August 15, 2013 1 Comment
Don’t want to bust anyone’s balloon. However, despite all the hoopla, Big Data and BI are still as Gartner has labeled it in the early stages of the Hype-Cycle. While the technology is approaching critical mass, the market is still hasn’t crossed the chasm. That is not to say vendors of such capabilities are full of puffery –actually not. The problem become one of Strategic Execution and Exploitation. Over the past ten+ years I have been actively engaged on and off on what arguably could be call BI and Big Data projects [Yup ten+ years!] What I’ve experienced from these projects is that for a first for IT, they’re ready before the line of business is to deploy the capability. Example: one instance, I developed an approach to infer what markets are heating up and which are cooling off. It took sales data, several other sources of information and some basic statistics -well may be a tiny bit more that basic statistics. The result was a heat map for geographic and industry market segments. The Marketing Executives were impress; not with what the data indicated, but with the fancy graphics. Clearly the message was lost in the media. A few years later I went back to check the forecasts, surprised by how close the predictions were. The sad aspect to this adventure, the system I spent time building before I left was lost to missing competency, thus the application was discarded as no one new how to use or what to use it for.
Fast forward several years. A colleague I was mentoring was asked to build BI capability for the company she just joined. Not a cynic by nature, I asked some fairly cynical questions:
- Do they know what they are going to do with the capabilities?
- Do they know what problems / questions they’re going to apply the capability to?
- If you build it, is there anyone that will know how to operate it?
- Does anyone have the background in statistics and analytics to understand the results?
The scary answer I got back was actually no across the board. The various departments had barely a remembrance of descriptive statistics. Concepts like correlation, confidence level, linear regression were vague labels from a statistics class they had years ago. It immediately drew me back to my engineering/manufacturing past and the comments attributed to Kelly Johnson (Lockheed Skunkworks): “I don’t need more computers, I just need more engineers that can read slide rules”. The comment infers that Kelly was anti-computer and that’s not really so. What I understood from his comment was you needed people that understood how the computer solved the problem and the range of where the solution should be, otherwise you’re at the mercy of the computer (i.e., The computer said…) As another wise pundit said “Computers make great slaves, but terrible masters”). Not understanding the underpinnings of how the capability works (not necessarily the computer logic) leaves you at the mercy of the computer. This not to say leaving the grunt work to the computer is bad, but to continue the old adage-feast I’m on “A fool with a Tool is Still a Fool” or “When all you have is a Hammer, everything looks like a nail”. Thus not all problems are suited to be solved by BI and/or Big Data, this is were Enterprise, Business Architects, and Solution Architects come into play.
Originally I was going to discuss BI and Big Data, however, in the end I’ve come to the conclusion the true value of Architects is not in the plans created to guide construction, but in the questions asked to ensure what is built solves the enterprise’s problems and provides true value. In ending If a capability is created and nobody uses it is it of value?