Structure in Threes: Metrics and Measurements

Spent the last few weeks divided between multiple projects and goals.  However, that gave me time to think about the topic of measurements.  Often in the IT field I see it being a great effort used to prove value or performance that ultimately has as its goal justifying existence.  When I hear about a new metrics initiative in most organizations it seems to boil down to how can we show with numbers that we’re meeting our targets.  All too often I think back to the radio show “The Prairie Home Companion” The closing words of the monologue are “Well, that’s the news from Lake Wobegon, where all the women are strong, all the men are good looking, and all the children are above average.”

This leads me to believe either we’ve set the bar too low or we’re measuring the wrong things.  About ten years ago I was at a senior leadership meeting at one of the well know technology powerhouses.  The one of the senior executives had, by edict, required a “serious” effort to measure performance.  The problem occurred not with the desire to measure, but the implementation of such.  There was no goal, other than measuring.  Thus each division, sub-division, unit, etc. identify metrics to report on.  In total over 1000 individual metrics of which cost the corporation lots of headcount to capture, collate, massage, and report at leadership meetings.  When asked if any of the these metrics were used to manage the organization or inform executives to make better decisions. I saw a lot of staring at shoes by people.  I asked why measure things if you’re not going to use the measurements to inform your decisions and directions; more shoe examinations.  Needless to say I was not invited back to a senior leadership meeting for a while.

Fast forward a few years later, the corporation is not doing as well as it once was and IT budgets are under fire.  I was brought back by a new leadership team; under cover.  That is I would dial in unadvertised and then consult to select members of the leadership team regarding my impressions and insights.  I think I was given this opportunity because during the previous incident I had predicted a scenario of what would happen to the IT org under its present course.  It was not a feat of great insight, it was merely a matter of connecting the dots. In either event I spent time with them, not specifically telling them what or how to measure, but why?  The old maxim “you get the behavior you measure” I’ve found almost true.  Often people measure what makes them look best.  Metrics are often tied to performance rewards.  This often eliminates these from being diagnostic tools.  This a little like the joke about the airplane captain announcing  on the PA system:  “I have bad new and good news.  The bad news is we’re lost. The good news is we’re making great time.”   Eventually there will be a crash.

Which brings me back to the metrics and measurements discussion.  Establishing a measurement program, one should first identify the strategic goals and objectives of the organization, then create metrics around those.  Next identify Key Performance Indicators (KPIs) that support those goals and objectives.  I look at KPIs a little differently than most I guess.  I think the label says it all these are “indicators” of performance, not the actual performance.  These are sort of like road sides telling you your goal is 100 miles ahead, then 50 miles, etc. and your speedometer it indicating 50 miles an hour.  Together these KPIs help you determine if you’re on the right path and operating at the right levels of efficiency and effectiveness.

Which again gets back to a metrics program.  A useful program has a mixture of “strategic” and “operational” metrics.  The strategic metrics tell you where your going, the operational metrics support you in letting you know you’re making progress towards getting to your goal with your resources.

IT Planning and Agile

Some of the new organizations I’m dealing with are going crazy with “Agile”.  The Hype Cycle  is in full force.  Soon Agile with be advertised to put the kids to bed on time, solve world peace, and feed the world.  The problem I see is not with Agile its with its close cousin “Addle” which is often what I see organizations implementing (I.e., the worst of each methodology glued together).   I will point out that I am neither a zealot for Waterfall or Agile; its a tool and like other power tools –notice the woodworking reference– it should be used carefully.  Too often I’ve seen Agile be used for an excuse for poor or no planning and/or poor development practices.

If one reads the original materials, nowhere does it say don’t plan.  Actually it indicates you need plans, just not at the level of precision and scope previously and wrongly drummed into people’s heads.  Agile suggests or recommends planning in short enough horizons such that the problem doesn’t change before you finishes planning (accuracy) and to a granularity (precision) that is enough to get the job done.  Another way to look at it:

If my problem was to cross the river and you spent five years planning to build a bridge, by the time you actually built the bridge I may have cross the river with a boat and now I’m faced with climbing a mountain (problem has changed).  If I want something to sit on I may want a chair, but it may not need to be designed and constructed to 1 millionth of an inch tolerance –which would cost a significant amount.  I could probably get by with 1/32 of an inch, produced at lower cost but yielding the same level of performance I desire.  So in the end Agile is a balancing act that I believe still requires planning; just different types of planning and a whole lot of thinking.

 

Knowledge Sharing

Spent the morning writing a summary of current project activity for a newsletter.  The team I’m on is relatively new and we’re just starting to establish processes and procedures for sharing knowledge and letting others in the organization know what we do and how we can help.   During writing my first draft I attempted to ensure it would not sound like some cold grey, boring status report; the kind you read and forget while you’re reading it.  While I’m writing this blog to generate content for a book I’m going to write this year –I don’t consider myself a writer, which is odd cause I’m such a voracious reader– it came to me that maybe more community wisdom would be shared if teams were required to write project histories in a blog rather than status reports.  Writing a project history entry seems to have me consider relating more aspect than: Did x, Cost y, Resulted in z or On schedule, On budget or project is Green, Yellow or Red status.

The history entry has me consider what we through was the problem and approach, what we discovered, how we adapted and what were the results.  If one looks at status reports these always seem to be the bare statistics not the story behind what happened and why.  I find that a terrible lost opportunity.  While I’ve not been a fan of autobiographies or biographies –I find talking to people about how they accomplished something or overcame an obstacle more insightful– as typically the Bios or at least the one’s I’ve read where more colorful status reports, not covering the deep insights and learnings.  This little morning insight may have me reconsider reading biographies a little deeper next time to see if the author really did but their insights down on paper.

Benefits Realization

Been a bit busy stoking the home fires of late.  Attended COFES, amazing conversations as usual.  This month I’ve been focused on several areas of applied business architecture research.  IT Portfolio Management, Benefits Management, and Complexity Management.  All three are related to my Structure in Threes project.

• I continue to develop the portfolio model section by section along with a working prototype.  Started considering the technology and system to offer to the market.
• Benefits Management this month is really a parallel track, both R&D for ensuring a portfolio action supports Enterprise Goals as well as applied practice for the projects I’m working on at Microsoft.  The past few weeks I’ve been creating a Benefits Dependency Network for one of the subprojects.  I’ll be reviewing and revising that today with stakeholders as well as creating a draft Benefits Management Plan to help ensure the initiatives realize the promised benefits.  Part of that will be a Results Chain Contribution Matrix, a Benefits Distribution Matrix, and a Stakeholder Management plan.  Most of these artifacts I’ll recommend to my group for future projects
• Complexity Management R&D is part of the BPR/M activities at work as well as Portfolio Management R&D.  Had a great discussion with Dr. Jacek Marczyk discussion elements of complexity.  We’ll have lots more to discuss.  I like his high level model:  Structure Elements x Uncertainty = Complexity.    I had previously separated Uncertainty from the equations I was developing:  Business Process Complexity =  {Information Complexity} x {Activity Complexity} using BPMN models as the base to calculate each factor.  As of yesterday I revised calculations from a standard node count bases to also include network linkages between nodes in each factor.   Later this week I’ll look at how I include Dr. Marczyk’s perspective of accounting for uncertainty.  I think I may also expand on that and use some of Courtney, Day, Schoemaker, and Primozic research into risk and uncertainty.  They’ve a lot of good materials that could apply to the problem space.

Structure in Threes: Building vs Planning

One of the problems systemic to the design and planning world is the often famous quote:  “Its done its just a small matter or implementation or programming or etc.”  All too often I’ve seen designers or planner consider the job done once the design is finished.  However, there is often a lot of work to get it accomplished; even it there are not changes to the design.  This got me thinking about the terminology used in regard to projects.  To me implementation take is a larger concept than just deployment which seems to be the typical thoughts around projects, especially in IT.  “The application has been installed we’re done”.  True implementation carries with it both deployment and adoption, anything less is just dumping.

The proposed remedy to this is to have adoption KPIs included in project metrics, not just development cost and schedule.  This would foster a greater concern that what is built is actually used.

Working Insights: Process Analysis presentation creation and the movies

Finished my presentation deck for Wednesday’s Business Architect call, sent it out to my manager & colleague, and other friends for comments.  The topic is business process analysis methods.  Its a short 20 minute presentation on the benefits and some methods of value for analyzing processes.  During developing I rediscovered what I already knew: Difficult to compress 2 hours of material into less than 20 minutes.  Makes me truly appreciate movie editors and directors…it take a lot of skill to cut things down to the bare minimum and no more.

Which has me reconsider my thoughts around BPR/M compliments I get in the form: “I’ve never seen anyone but you be able to diagram a process on the fly while we’re discussing it” or “How can you trim these processes down so well that there are minimum steps and each one contributes real value so easily?”  Often I just smile and say thank you, but really don’t think much of it its just what I do.  But I guess its because I’ve spent years honing my craft like editors and movie directors learning how to cut away what’s non necessary to the accomplish the job; in once case telling a story and in mine making a highly efficient and effective process for an enterprise.  I’m sure by now my colleague Sarah is smiling, nodding her head, and saying of course.

Unified Business Modeling and Architecture Capability: Using a common language across multiple business tools

Started building a Capacity Planning Model to evaluate processes I’m modeling in BPMN.  Contacted a colleague who shares Model-Driven process execution vision with the idea to build a capacity and financial modeling capability add-on to his execution engine.  I can see a real benefit in the ability to model, analyze and execute processes using the same graphical language.  Currently the tool suite for Business Architects is a cobbling of various tools which means having to do multiple translations and transcriptions from one system to another.  The results of which is a lost of productivity, agility, increased configuration management overhead and reduced benefits to the business.  This is similar to the problem programmer have had with using multiple tools not meant to work together.  What’s needed for the business architect is a common tool suite or Framework that integrates the various models and variables making them available for multiple uses; Different types of Analysis, Execution, Monitoring and Reporting.  I had this vision years ago during my CIO Workbench / Activity Directory Conceptual Design days.  Now maybe the time to build it

Insights: Meetings

Occasionally I have the luxury of sitting back and watching project meetings as an FYI attendee rather than a active participant.  During a recent meeting that was to set context for a direction change and gain buy-in from stakeholders I watched as the two meeting leads followed a careful script crafted the previous day gradually spun off track.  These were two seasoned professionals with more than a few years experience.  People I highly respect. So what happened?

Like so many other group meetings, both the pre-meeting developing the script for the buy-in meeting and the buy-in meeting,  understanding the roles at the table and there agendas was not prime on the mines of the participants.  Often in meetings I find we –and I’ll include myself in this pattern– are more interested in our own specific agenda than achievement of the overall goal of the team or scoring points in intellectual arguments on detail procedures rather than outcomes.  While the pre-meeting had established its goal of gaining buy-in from the stakeholders the approach to gaining such was less thought out.  The team spent time creating a logical agenda to present what and why we were going to do differently.  This was done optimistically or blindly in the believed if the logic was correct we’d get buy-in.  While we got some agreement, the buy-in meeting slowly drifted away as stakeholders 1) raised issues that were not covered in the presentation but were of importance to them and 2) had a different understanding of terms, concepts, methods and how these are combined for results.  Even the outcomes expected generated controversy.  The goals of each stakeholders were different than the goals of the project or rather achievement of these stakeholder goals were not necessarily supportive of the overall goal of the project.  That is not to say these goals were counter to achievement of the project, but these needed to be aligned in context of the overall project.

Which brings me back to the corrective action we should have taken during the pre-meeting.  Years ago I brought to the attention of my management an article in Sloan Management Review : A Framework for Managing IT-Enabled Change,  Summer 1993 July 15, 1993 Robert I. Benjamin and Eliot Levinson.   I was promptly told to bury the article, we don’t do that here in this company.  Such effort would be seen as manipulation rather than good leadership practices.

In that article they had an approach towards visually planning gaining stakeholder buy-in which would have served us well.  However, while some progress in language regarding gaining stakeholder buy-in has improved in the IT profession, considered focus and concern in that direction are still merely a follow the numbers activity.  Would the meeting have been different, more productive, and gained agreement -no commitment- faster had we used this approach.  I don’t know for sure.  I expect it would as it had worked for me very well at other companies, but I would have like to have seen it tried.

The other insight gained from the meeting was really a reminder that terms and concepts are not always understood by everyone in the same manner  These are influenced by the context and role people play within a particular situation. This is especially true in the business process domain which is very abstract and often leans itself to multiple interpretations to terms, many times drifting off as a new label to old ways of doing business rather than the new approach it was suppose to be.

 

Process Modeling

My Thursday walkthrough with my team reinforced best practices learned years ago:

1. Level of detail needed is in the eye of the beholder
2. Modeling is iterative process
3. Partitioning any system or process is more art than science, expect when it comes to consistently applying the decomposition rules consistently

Walkthrough went very well for the amount of time and data available.  Looking forward to getting business community’s feedback.  I hope to have lots of rocks thrown at the model to insure it reflects what they do or want to do.  I had considered pulling one of my modeling tricks to get involvement [adding a few mistakes on purpose] however, I don’t think getting feedback from the business is going to be an issue.  They seem very engaged and willing to comment.

Structure in Threes: Management Systems

For the record I’m typically for performance based compensation systems.  Maybe because I think you should be focused to perform your role for the best of the company to start with.  I see approaches such as stack ranking, top grading, etc. as creating a cascading effect that has employees and management “game” the system.  The end result is a death spiral where people spend more time looking good than doing good at the expense of the customer and investors.  After reading chapter eight of Rummler’s book reDiscovering Value, I see why and where these systems have gone wrong; basically measuring and rewarding the wrong behavior (local optimization) rather than doing the best to create value across the enterprise.

I’m finding a lot to like about Rummler’s approach towards creating a management system that works:

• First reengineer the processes cross functionally to enable real value creation
• Second design a dual management system that focuses on the value creation performance and then resource performance
• Third design the compensation system weighted towards value creation system performance as top priority

I can see this approach keeping in check the desire to optimize resources in deference to creating value which is what keeps a business in business.