-
-
Marketplace
-
Channel Resources
Articles from this Site
-
One brand, one Web site! DM Review is now the home of all the content you're used to at BIReview.com and much more. If you are registered at BIReview.com, you're already registered at DM Review. If not, take just a moment to sign up for all the free services we have for you at the new DMReview.com.
The Limits of Measurement
Measurement is at the core of Performance Management, but is it really possible to use measurement - or metrics in the current parlance - to drive an organization? There are two points of view, one widely accepted and current, the other opposing and more abstract.
The conventional wisdom is that our technology is perfectly capable of providing detailed, current and relevant performance information to stakeholders in an enterprise, including executives, managers, functional people, customers, vendors and regulators. Because we are blessed with abundant computing resources, connectivity, bandwidth and even standards, it is possible to present this information in cognitively effective ways (dashboards, for example). Recipients are able to receive the information in the manner in which they choose and the whole process pays dividends by supporting the notion that, "If you can't measure it, you can't manage it." It is hard to imagine how anyone could manage a large undertaking without measurement, isn't it? And most presentations I've heard quickly stress that measurement is only part of the solution. The first step is knowing what to measure; then measuring it accurately; then finding a way to disseminate the information for maximum impact, (figuring out how to keep it current and relevant); and then being able to actually do something about the results. A different way of saying this is that technology is never a solution to social problems, and interactions between human beings are inherently social. This is why Performance Management is a very complex discipline, not just the implementation of dashboard or scorecard technology. Luckily, the business community seems to be plugged into this concept in a way they never were in the old context of Business Intelligence. In this new context, organizations understand that measurement tools only imply remediation and that Business Intelligence is most often applied merely to inform people, not to catalyze change. In practice, such undertakings almost always lack a change management methodology or portfolio.
But there is an argument against measurement too. Unlike machines or chemical reactions in a beaker, human beings are aware that they are being measured. In the realm of physics, Heisenberg's Uncertainty Principle demonstrates that the act of measurement itself can very often distort the phenomena one is attempting to measure. When it comes to sub-atomic particles, we can pretty much assume it is a physical law that underlies this behavior. With people, the unseen subtext is clearly conscious. People find the most ingenious ways to distort measurement systems to generate the numbers that are desired, not only NOT providing the desired behaviors, often becoming more dysfunctional through the effort. There are excellent, documented examples of this in Transforming the Bottom Line, by Hope and Hope. The authors' contention is that measurement of people always introduces distortion and often brings dysfunction because measurement is never more than a proxy or an approximation of the real phenomena. We all know that incomplete proxies are too easy to exploit in the same way that inadequate software with programming gaps beckon to unscrupulous hackers. However, one doesn't have to be malicious to subvert a measurement system. After all, voluntary compliance to the tax code encourages a national obsession with "loopholes," and what salesman hasn't "sandbagged" a few deals for next quarter after she's met her quota for the current one?
The solution is not to discard measurement (Hope and Hope are accountants after all). Instead, they suggest being conscious of this tendency and to be vigilant and thorough in the design of measurement systems. We all have a tendency toward simplifying things but in some cases it appears better to not measure at all than to produce something inadequate. Performance management, to achieve its goals, has to be applied effectively, which is to say, with superior execution of technology, implementation and management. It has to be designed to be responsive to both incremental and unpredicted changes in the organization and the environment. There are no roadmaps for this. This is truly the first time that analytical and measurement technology can be embedded in day-to-day, instantaneous decision-making and tracking, and the industry is sorely lacking in skills and experience to pull it off. Those organizations that have been successful so far have relied on existing methodologies (balanced scorecard, for example) to guide them through the more uncertain steps of metric formulation and change management to close the loop.
The question of whether you can ever adequately measure an organization is still open. To the extent that there are statutory and regulatory requirements, such as taxation, SEC or specific industry regulations, the answer is clearly yes. But those measurements are dictated. To measure performance after the fact, at aggregated levels, is only useful to a point. The closer and closer a measurement system gets to the actual events and actions that drive the higher-level numbers, the less reliable the cause-effect relationship becomes, just like Heisenberg found so long ago. There are many examples in the management literature of everyone "doing the right thing" while the wheels are coming off the organization.
Raden's Recommended Reading
- Transforming the Bottom Line: Managing Performance with the Real Numbers, Tony Hope and Jeremy Hope, Boston: Harvard Business School Press, 1995
- In the Age of the Smart Machine: The Future of Work and Power, Shoshana Zuboff, New York, Basic Books, 1988
- "The New Productivity Challenge," Peter Drucker, Harvard Business Review, (November-December 1991: 70.
- "The Information Executives Truly Need," Peter Drucker, Harvard Business Review, January-February 1995: 54-62
Neil Raden is the founder of Hired Brains, Inc., http://www.hiredbrains.com. Hired Brains provides consulting, systems integration and implementation services in business intelligence, decision automation and business process integration for clients worldwide. Hired Brains Research provides consulting, market research, product marketing and advisory services to the software ndustry. Based in Santa Barbara, CA, Raden is an active consultant and widely published author and speaker. He welcomes your comments at nraden@hiredbrains.com.
For more information on related topics, visit the following channels:
