Structure in Threes: Metrics and Measurements

Spent the last few weeks divided between multiple projects and goals.  However, that gave me time to think about the topic of measurements.  Often in the IT field I see it being a great effort used to prove value or performance that ultimately has as its goal justifying existence.  When I hear about a new metrics initiative in most organizations it seems to boil down to how can we show with numbers that we’re meeting our targets.  All too often I think back to the radio show “The Prairie Home Companion” The closing words of the monologue are “Well, that’s the news from Lake Wobegon, where all the women are strong, all the men are good looking, and all the children are above average.”

This leads me to believe either we’ve set the bar too low or we’re measuring the wrong things.  About ten years ago I was at a senior leadership meeting at one of the well know technology powerhouses.  The one of the senior executives had, by edict, required a “serious” effort to measure performance.  The problem occurred not with the desire to measure, but the implementation of such.  There was no goal, other than measuring.  Thus each division, sub-division, unit, etc. identify metrics to report on.  In total over 1000 individual metrics of which cost the corporation lots of headcount to capture, collate, massage, and report at leadership meetings.  When asked if any of the these metrics were used to manage the organization or inform executives to make better decisions. I saw a lot of staring at shoes by people.  I asked why measure things if you’re not going to use the measurements to inform your decisions and directions; more shoe examinations.  Needless to say I was not invited back to a senior leadership meeting for a while.

Fast forward a few years later, the corporation is not doing as well as it once was and IT budgets are under fire.  I was brought back by a new leadership team; under cover.  That is I would dial in unadvertised and then consult to select members of the leadership team regarding my impressions and insights.  I think I was given this opportunity because during the previous incident I had predicted a scenario of what would happen to the IT org under its present course.  It was not a feat of great insight, it was merely a matter of connecting the dots. In either event I spent time with them, not specifically telling them what or how to measure, but why?  The old maxim “you get the behavior you measure” I’ve found almost true.  Often people measure what makes them look best.  Metrics are often tied to performance rewards.  This often eliminates these from being diagnostic tools.  This a little like the joke about the airplane captain announcing  on the PA system:  “I have bad new and good news.  The bad news is we’re lost. The good news is we’re making great time.”   Eventually there will be a crash.

Which brings me back to the metrics and measurements discussion.  Establishing a measurement program, one should first identify the strategic goals and objectives of the organization, then create metrics around those.  Next identify Key Performance Indicators (KPIs) that support those goals and objectives.  I look at KPIs a little differently than most I guess.  I think the label says it all these are “indicators” of performance, not the actual performance.  These are sort of like road sides telling you your goal is 100 miles ahead, then 50 miles, etc. and your speedometer it indicating 50 miles an hour.  Together these KPIs help you determine if you’re on the right path and operating at the right levels of efficiency and effectiveness.

Which again gets back to a metrics program.  A useful program has a mixture of “strategic” and “operational” metrics.  The strategic metrics tell you where your going, the operational metrics support you in letting you know you’re making progress towards getting to your goal with your resources.

IT Planning and Agile

Some of the new organizations I’m dealing with are going crazy with “Agile”.  The Hype Cycle  is in full force.  Soon Agile with be advertised to put the kids to bed on time, solve world peace, and feed the world.  The problem I see is not with Agile its with its close cousin “Addle” which is often what I see organizations implementing (I.e., the worst of each methodology glued together).   I will point out that I am neither a zealot for Waterfall or Agile; its a tool and like other power tools –notice the woodworking reference– it should be used carefully.  Too often I’ve seen Agile be used for an excuse for poor or no planning and/or poor development practices.

If one reads the original materials, nowhere does it say don’t plan.  Actually it indicates you need plans, just not at the level of precision and scope previously and wrongly drummed into people’s heads.  Agile suggests or recommends planning in short enough horizons such that the problem doesn’t change before you finishes planning (accuracy) and to a granularity (precision) that is enough to get the job done.  Another way to look at it:

If my problem was to cross the river and you spent five years planning to build a bridge, by the time you actually built the bridge I may have cross the river with a boat and now I’m faced with climbing a mountain (problem has changed).  If I want something to sit on I may want a chair, but it may not need to be designed and constructed to 1 millionth of an inch tolerance –which would cost a significant amount.  I could probably get by with 1/32 of an inch, produced at lower cost but yielding the same level of performance I desire.  So in the end Agile is a balancing act that I believe still requires planning; just different types of planning and a whole lot of thinking.


Knowledge Sharing

Spent the morning writing a summary of current project activity for a newsletter.  The team I’m on is relatively new and we’re just starting to establish processes and procedures for sharing knowledge and letting others in the organization know what we do and how we can help.   During writing my first draft I attempted to ensure it would not sound like some cold grey, boring status report; the kind you read and forget while you’re reading it.  While I’m writing this blog to generate content for a book I’m going to write this year –I don’t consider myself a writer, which is odd cause I’m such a voracious reader– it came to me that maybe more community wisdom would be shared if teams were required to write project histories in a blog rather than status reports.  Writing a project history entry seems to have me consider relating more aspect than: Did x, Cost y, Resulted in z or On schedule, On budget or project is Green, Yellow or Red status.

The history entry has me consider what we through was the problem and approach, what we discovered, how we adapted and what were the results.  If one looks at status reports these always seem to be the bare statistics not the story behind what happened and why.  I find that a terrible lost opportunity.  While I’ve not been a fan of autobiographies or biographies –I find talking to people about how they accomplished something or overcame an obstacle more insightful– as typically the Bios or at least the one’s I’ve read where more colorful status reports, not covering the deep insights and learnings.  This little morning insight may have me reconsider reading biographies a little deeper next time to see if the author really did but their insights down on paper.

Benefits Realization

Been a bit busy stoking the home fires of late.  Attended COFES, amazing conversations as usual.  This month I’ve been focused on several areas of applied business architecture research.  IT Portfolio Management, Benefits Management, and Complexity Management.  All three are related to my Structure in Threes project.

  • I continue to develop the portfolio model section by section along with a working prototype.  Started considering the technology and system to offer to the market.
  • Benefits Management this month is really a parallel track, both R&D for ensuring a portfolio action supports Enterprise Goals as well as applied practice for the projects I’m working on at Microsoft.  The past few weeks I’ve been creating a Benefits Dependency Network for one of the subprojects.  I’ll be reviewing and revising that today with stakeholders as well as creating a draft Benefits Management Plan to help ensure the initiatives realize the promised benefits.  Part of that will be a Results Chain Contribution Matrix, a Benefits Distribution Matrix, and a Stakeholder Management plan.  Most of these artifacts I’ll recommend to my group for future projects
  • Complexity Management R&D is part of the BPR/M activities at work as well as Portfolio Management R&D.  Had a great discussion with Dr. Jacek Marczyk discussion elements of complexity.  We’ll have lots more to discuss.  I like his high level model:  Structure Elements x Uncertainty = Complexity.    I had previously separated Uncertainty from the equations I was developing:  Business Process Complexity =  {Information Complexity} x {Activity Complexity} using BPMN models as the base to calculate each factor.  As of yesterday I revised calculations from a standard node count bases to also include network linkages between nodes in each factor.   Later this week I’ll look at how I include Dr. Marczyk’s perspective of accounting for uncertainty.  I think I may also expand on that and use some of Courtney, Day, Schoemaker, and Primozic research into risk and uncertainty.  They’ve a lot of good materials that could apply to the problem space.

Structure in Threes: Building vs Planning

One of the problems systemic to the design and planning world is the often famous quote:  “Its done its just a small matter or implementation or programming or etc.”  All too often I’ve seen designers or planner consider the job done once the design is finished.  However, there is often a lot of work to get it accomplished; even it there are not changes to the design.  This got me thinking about the terminology used in regard to projects.  To me implementation take is a larger concept than just deployment which seems to be the typical thoughts around projects, especially in IT.  “The application has been installed we’re done”.  True implementation carries with it both deployment and adoption, anything less is just dumping.

The proposed remedy to this is to have adoption KPIs included in project metrics, not just development cost and schedule.  This would foster a greater concern that what is built is actually used.

Working Insights: Process Analysis presentation creation and the movies

Finished my presentation deck for Wednesday’s Business Architect call, sent it out to my manager & colleague, and other friends for comments.  The topic is business process analysis methods.  Its a short 20 minute presentation on the benefits and some methods of value for analyzing processes.  During developing I rediscovered what I already knew: Difficult to compress 2 hours of material into less than 20 minutes.  Makes me truly appreciate movie editors and directors…it take a lot of skill to cut things down to the bare minimum and no more.

Which has me reconsider my thoughts around BPR/M compliments I get in the form: “I’ve never seen anyone but you be able to diagram a process on the fly while we’re discussing it” or “How can you trim these processes down so well that there are minimum steps and each one contributes real value so easily?”  Often I just smile and say thank you, but really don’t think much of it its just what I do.  But I guess its because I’ve spent years honing my craft like editors and movie directors learning how to cut away what’s non necessary to the accomplish the job; in once case telling a story and in mine making a highly efficient and effective process for an enterprise.  I’m sure by now my colleague Sarah is smiling, nodding her head, and saying of course.

Unified Business Modeling and Architecture Capability: Using a common language across multiple business tools

Started building a Capacity Planning Model to evaluate processes I’m modeling in BPMN.  Contacted a colleague who shares Model-Driven process execution vision with the idea to build a capacity and financial modeling capability add-on to his execution engine.  I can see a real benefit in the ability to model, analyze and execute processes using the same graphical language.  Currently the tool suite for Business Architects is a cobbling of various tools which means having to do multiple translations and transcriptions from one system to another.  The results of which is a lost of productivity, agility, increased configuration management overhead and reduced benefits to the business.  This is similar to the problem programmer have had with using multiple tools not meant to work together.  What’s needed for the business architect is a common tool suite or Framework that integrates the various models and variables making them available for multiple uses; Different types of Analysis, Execution, Monitoring and Reporting.  I had this vision years ago during my CIO Workbench / Activity Directory Conceptual Design days.  Now maybe the time to build it