The rules of business scorecarding: Garbage in, garbage out

During the recent Project Management Institute conference in Toronto, one of the Microsoft Project product managers from Redmond gave a presentation on a soon-to-be-released product called Business Scorecard Manager. The product, which is in its final stages of beta testing, provides a central point to create formulas and displays for key performance indicators (KPIs), which are then made into a variety of context-sensitive scorecards or visible displays. The demonstration showed data from a variety of sources — financial, sales and even geographic — were woven into a single display.
The icing on this demo was how the project data from Microsoft Project Server was woven into the Business Scorecard display. The whole structure sits within Microsoft’s Sharepoint Portal Server technology.
I’ve been doing some investigation of scorecard systems since I wrote about it in this column over a year ago. Microsoft’s is not the first scorecard product on the market, but I found it interesting that the movement has picked up enough momentum that Microsoft felt compelled to launch its own offering.
After the presentation, I was approached by several attendees who discussed the relevance of using scorecarding with enterprise project management and of course, I reiterated my enthusiasm from a couple of years ago. One comment in particular, however, brought me up short.
If the underlying data is of questionable quality, said one of my colleagues, wouldn’t that make the entire system suspect? The more I think about that, the more it disturbs me.
We depend in great part on the natural human filtering that goes on when data is examined and entered. In systems that have an extensive legacy culture, such as core financial systems, we accept the basis of how data is entered because it is common to all organizations.
It is when we apply the same notion to other enterprise systems that we often see how challenging it is to establish stability in this type of data. CRM, enterprise project management and HR systems all require a remarkable amount of effort to make their data as trustworthy as core financials.
It’s not the technology, of course, it’s the standard for using that technology that must be adapted with sufficient uniformity to make sense. In my own firm, we’re currently struggling with CRM terms for what makes a lead versus a contact versus an opportunity.
When we create an executive display system to roll all that data up to an easy-to-read, one-page display on which core business decisions are made, we have to take pause.
I’m sure executive dashboards of scorecard type information will be hot sellers, but before we deliver a system that eliminates the need for human filtering of data, we have to make sure the underlying source data for those displays delivers what we need.