Structure in Threes: Metrics and Measurements

Spent the last few weeks divided between multiple projects and goals.  However, that gave me time to think about the topic of measurements.  Often in the IT field I see it being a great effort used to prove value or performance that ultimately has as its goal justifying existence.  When I hear about a new metrics initiative in most organizations it seems to boil down to how can we show with numbers that we’re meeting our targets.  All too often I think back to the radio show “The Prairie Home Companion” The closing words of the monologue are “Well, that’s the news from Lake Wobegon, where all the women are strong, all the men are good looking, and all the children are above average.”

This leads me to believe either we’ve set the bar too low or we’re measuring the wrong things.  About ten years ago I was at a senior leadership meeting at one of the well know technology powerhouses.  The one of the senior executives had, by edict, required a “serious” effort to measure performance.  The problem occurred not with the desire to measure, but the implementation of such.  There was no goal, other than measuring.  Thus each division, sub-division, unit, etc. identify metrics to report on.  In total over 1000 individual metrics of which cost the corporation lots of headcount to capture, collate, massage, and report at leadership meetings.  When asked if any of the these metrics were used to manage the organization or inform executives to make better decisions. I saw a lot of staring at shoes by people.  I asked why measure things if you’re not going to use the measurements to inform your decisions and directions; more shoe examinations.  Needless to say I was not invited back to a senior leadership meeting for a while.

Fast forward a few years later, the corporation is not doing as well as it once was and IT budgets are under fire.  I was brought back by a new leadership team; under cover.  That is I would dial in unadvertised and then consult to select members of the leadership team regarding my impressions and insights.  I think I was given this opportunity because during the previous incident I had predicted a scenario of what would happen to the IT org under its present course.  It was not a feat of great insight, it was merely a matter of connecting the dots. In either event I spent time with them, not specifically telling them what or how to measure, but why?  The old maxim “you get the behavior you measure” I’ve found almost true.  Often people measure what makes them look best.  Metrics are often tied to performance rewards.  This often eliminates these from being diagnostic tools.  This a little like the joke about the airplane captain announcing  on the PA system:  “I have bad new and good news.  The bad news is we’re lost. The good news is we’re making great time.”   Eventually there will be a crash.

Which brings me back to the metrics and measurements discussion.  Establishing a measurement program, one should first identify the strategic goals and objectives of the organization, then create metrics around those.  Next identify Key Performance Indicators (KPIs) that support those goals and objectives.  I look at KPIs a little differently than most I guess.  I think the label says it all these are “indicators” of performance, not the actual performance.  These are sort of like road sides telling you your goal is 100 miles ahead, then 50 miles, etc. and your speedometer it indicating 50 miles an hour.  Together these KPIs help you determine if you’re on the right path and operating at the right levels of efficiency and effectiveness.

Which again gets back to a metrics program.  A useful program has a mixture of “strategic” and “operational” metrics.  The strategic metrics tell you where your going, the operational metrics support you in letting you know you’re making progress towards getting to your goal with your resources.

IT Planning and Agile

Some of the new organizations I’m dealing with are going crazy with “Agile”.  The Hype Cycle  is in full force.  Soon Agile with be advertised to put the kids to bed on time, solve world peace, and feed the world.  The problem I see is not with Agile its with its close cousin “Addle” which is often what I see organizations implementing (I.e., the worst of each methodology glued together).   I will point out that I am neither a zealot for Waterfall or Agile; its a tool and like other power tools –notice the woodworking reference– it should be used carefully.  Too often I’ve seen Agile be used for an excuse for poor or no planning and/or poor development practices.

If one reads the original materials, nowhere does it say don’t plan.  Actually it indicates you need plans, just not at the level of precision and scope previously and wrongly drummed into people’s heads.  Agile suggests or recommends planning in short enough horizons such that the problem doesn’t change before you finishes planning (accuracy) and to a granularity (precision) that is enough to get the job done.  Another way to look at it:

If my problem was to cross the river and you spent five years planning to build a bridge, by the time you actually built the bridge I may have cross the river with a boat and now I’m faced with climbing a mountain (problem has changed).  If I want something to sit on I may want a chair, but it may not need to be designed and constructed to 1 millionth of an inch tolerance –which would cost a significant amount.  I could probably get by with 1/32 of an inch, produced at lower cost but yielding the same level of performance I desire.  So in the end Agile is a balancing act that I believe still requires planning; just different types of planning and a whole lot of thinking.

 

Modern IT Portfolio Management: Risk: System Definition

System Definitional Risk

One of the serious risks in Architect side is system definition. This is partitioned into completeness(e.g., maturity level [none to integrated], Validation level [none, author’s bench check……Simulation of requirements], and requirements freshness (i.e., things change, was what was needed specified and validated too long ago….).

 

 

Benefits Realization

Been a bit busy stoking the home fires of late.  Attended COFES, amazing conversations as usual.  This month I’ve been focused on several areas of applied business architecture research.  IT Portfolio Management, Benefits Management, and Complexity Management.  All three are related to my Structure in Threes project.

  • I continue to develop the portfolio model section by section along with a working prototype.  Started considering the technology and system to offer to the market.
  • Benefits Management this month is really a parallel track, both R&D for ensuring a portfolio action supports Enterprise Goals as well as applied practice for the projects I’m working on at Microsoft.  The past few weeks I’ve been creating a Benefits Dependency Network for one of the subprojects.  I’ll be reviewing and revising that today with stakeholders as well as creating a draft Benefits Management Plan to help ensure the initiatives realize the promised benefits.  Part of that will be a Results Chain Contribution Matrix, a Benefits Distribution Matrix, and a Stakeholder Management plan.  Most of these artifacts I’ll recommend to my group for future projects
  • Complexity Management R&D is part of the BPR/M activities at work as well as Portfolio Management R&D.  Had a great discussion with Dr. Jacek Marczyk discussion elements of complexity.  We’ll have lots more to discuss.  I like his high level model:  Structure Elements x Uncertainty = Complexity.    I had previously separated Uncertainty from the equations I was developing:  Business Process Complexity =  {Information Complexity} x {Activity Complexity} using BPMN models as the base to calculate each factor.  As of yesterday I revised calculations from a standard node count bases to also include network linkages between nodes in each factor.   Later this week I’ll look at how I include Dr. Marczyk’s perspective of accounting for uncertainty.  I think I may also expand on that and use some of Courtney, Day, Schoemaker, and Primozic research into risk and uncertainty.  They’ve a lot of good materials that could apply to the problem space.

Structure in Threes: Budgeting and Planning -observations and muses

The beatings will continue until moral improves, or so goes the typical planning cycle each year.  Although being an observer of the process in many enterprises for 30 years and a unwilling participant at times, I would classify these activities as anything but planning.  These are more like the High School Senior Prom with all the politics and drama leading up to the final ceremonies.  Management spends hours of their as well as staff time building a case to justify their groups existence rather than actually looking at how and what to contribute to the enterprise’s bottom like.  Then having to “re-plan”  funding / headcount don’t match political expectations.

It doesn’t have to be this war dance each year.  Developing a real enterprise portfolio management system could remedy this, however, methodology zealots (aka methodologist, of which I’m sad to say I associate myself with) often get caught up in the minutia rather than the goes.  A joke I’m fond of telling during presentations when asked about strict adherence to any methodology might clue you into my worries about methodologies that too often become dogma:  What’s the difference between a terrorist and a methodologist? Answer: You can negotiate with a terrorist.

Much of the research and design work I’ve done over the years is now starting to converge:  Process Modeling (IDEF0 & BPMN),  Simulation and Analysis of systems (System Dynamics and Viable Systems Models), Digital Nervous System (AD), IT Economics ( ISIS, Information Economic/BEAM, REJ, and VRF) and now integrating financial Portfolio Management Concepts with VA/VE and Systems Engineering Concepts with the focus on designing enterprises like one would design any other product or service.  As I near competition of my basic research for Modern IT Portfolio Management I look forward to thinking about how to deploy and enable adoption.  Now that the basics of a digital nervous system are in place, its time to create the “brain of the firm “which I see is a portfolio management system connected to the DNS driven by people making fact based decisions.

The one caveat to using this approach is taking it too far, enterprises are composed of people as are markets.  And people are emotional, thus making decisions that address attempts at gratification for those emotions.  Thus Portfolio management needs to be applied with flexibility to not only accommodate this phenomena but exploit it in the positive sense.  Then perhaps budgeting and planning will be more of a constructive than destructive activity, creating intrapreuers in the company.

Structure in Threes: Building vs Planning

One of the problems systemic to the design and planning world is the often famous quote:  “Its done its just a small matter or implementation or programming or etc.”  All too often I’ve seen designers or planner consider the job done once the design is finished.  However, there is often a lot of work to get it accomplished; even it there are not changes to the design.  This got me thinking about the terminology used in regard to projects.  To me implementation take is a larger concept than just deployment which seems to be the typical thoughts around projects, especially in IT.  “The application has been installed we’re done”.  True implementation carries with it both deployment and adoption, anything less is just dumping.

The proposed remedy to this is to have adoption KPIs included in project metrics, not just development cost and schedule.  This would foster a greater concern that what is built is actually used.

Today’s Applied Research Agenda

Today’s agenda: R&D around Capacity Management for Services and Processes.  My presentation on metrics and measurement for processes went well, but I know risk and capacity management are a significant lack in this field.  Somehow everyone has gotten the opinion that Moore’s Law will bail them out…that hope is what the other engineering disciplines know now leads to unsustainability.  Received old book I ordered from Amazon Market [Computer Systems Performance Modeling, 1981 –Sauer]   that covered some of this in regards to computer systems.  Coupled with Meadows Limits to Growth (Systems Dynamics that Forrester introduced) I believe I can develop an approach to monitor, measure, and manage processes in more than a reactive way currently the industry norm.  While it will not likely be Nobel Prize winning stuff, I think it will help make Microsoft more competitive and responsive to customer needs