What’s your IQ? It’s not a facetious trivial question in this age of enterprise data integration and one-to-one marketing. Information quality is one of those issues that data-addicted companies would rather not think about. A recent Forrester Research study found that 20 per cent of companies had
no consistent IQ strategy at all, and many of the rest deal with the issue on an ad-hoc basis.
“”Data quality really is business’s dirty secret,”” says Frank Dravis, vice-president of information quality at First Logic, a consultancy specializing in IQ. “”You don’t find a lot of ROI stories about it because no one wants to admit they have a problem.””
Yet everyone seems to agree that accurate, timely, actionable data is one of the keys to success in of 21st Century business.
“”It is very important to us,”” says Jean-William Milon, a business manager in database marketing at Microcell. “”Reliable data is the foundation of CRM and customer knowledge, and that is extremely important, especially in a subscription-based business. This is a very competitive market, and we have to ensure that we have a competitive edge at all times.””
The issue isn’t new, says SAP Canada national marketing manager Mike Davidson. What is new is that the demands and uses for enterprise data have created a situation in which companies can no longer stumble around, manually picking out errors as they go.
“”What has happened over the last five years is that systems are being put in place that provide data transparency across the enterprise,”” Davidson says. “”That’s what has made it an enterprise issue.””
Bad data can be a serious kick to the bottom line. According to the Data Warehousing Institute, lost revenue due to poor quality information totals as much as $600 million (US) a year in the United States.
Of course, it’s all out of sight, out of mind until someone actually needs the data. The biggest push for information quality is coming out of corporate marketing departments. With the pressure on to create automated and effective CRM systems and crank up the effectiveness of one-to-one sales, the marketing guys know only too well that, at the end of the day, they have to know whom they’re selling to.
The problem is that enterprise data can become tainted from the first moment that it’s captured. Stephen Langdell, an Oxford, UK-based scientist with the National Algorithms Group and one of the developers of NAG’s data mining and cleaning components, says that there are two principle scenarios of data corruption.
“”In the first case, you might simply be missing data,”” he says. “”This would usually be fairly obvious. The big question is whether you can replace the missing data.””
In the second scenario, enterprise data is fairly complete, but it’s corrupt or inaccurate. “”This can happen in so many ways,”” he says. “”It could be that the data was entered incorrectly, either by data entry personnel or by a customer in a Web form, and it could also be due to things like the fraudulent use of credit cards.”” In some cases, companies can have many duplicate records of the same data, due to corporate mergers or the recent integration of departmental systems into enterprise applications.
Most often, however, data simply goes bad thanks to the normal churn of business. “”Data about people and products is tremendously dynamic,”” Dravis says. “”We age, we get promotions, phone numbers and addresses change. And forget about people — look at product churn. Even if you start with perfect data today, it will be imperfect by tomorrow.””
Given the nature of the problem, it can only get worse. According to Langdell, companies “”are pretty much in denial,”” but he notes that information quality is rarely a hopeless situation.
A hard, honest look
The first step is an enterprise-wide information quality assessment, where a company sits and takes a hard, honest look at its information quality. Dravis notes that this can be a difficult process of soul-searching, where individual managers have to put the interests of the enterprise above their departments. “”No one likes to admit that they have a problem,”” he says, “”but the truth is that everyone has a problem.””
Langdell adds that some kind of outside perspective is usually necessary. While it’s possible to run data quality applications from internal business models, Langdell points out that enterprises can sometimes find themselves down the rabbit hole in the Alice-in-Wonderland situation of assessing the quality of their data using models derived from the data they are assessing.
Having said that, there are clear inflection points when a company has to consider cleansing its data. One of the most dramatic is the aftermath of a merger, when two companies of dissimilar corporate cultures try to march into the future as a single entity. The problem is that, by blending two data systems, they open themselves up to duplication and the chaos of dissimilar formats.
“”Merger and acquisition is a radical driver for information quality,”” Dravis says. “”Some companies linger on in the future of maintaining two IT systems, but then they don’t get the full advantage of the merger. Merging companies must mean householding and standardizing data.””
A less radical and more practical approach is to ensure data quality from the outset, when you set up your enterprise data system. If garbage comes out where garbage goes in, then make sure your data isn’t garbage from Day 1, says SAP’s Davidson.
“”We tell new SAP customers that they have to make their data as clean as possible before it gets into the system,”” he says. “”For existing customers, a lot of the work is going back and looking at how to cleanse it. You have to go back to the drawing board to fix it.””
Don’t make the mistake that information quality ends there. Once your data is clean, you have to keep it that way, and correct all of the inconsistencies that creep in from the mundane churn of daily life.
Starting Small
For a company like Microcell, that means doing everything from verifying subscribers’ contact and account information at regular intervals to a constant technological oversight of what Milon calls “”our biggest asset.””
Dravis likes to think of the ongoing process as a cycle that begins with data profiling, to find the problems and data cleansing to eliminate them, and then continues on to data enhancement and consolidation before beginning the cycle again. It’s a complex and substantial process, but it’s necessary, he says. Nevertheless, Dravis is quick to point out that the size of a enterprise-wide information quality process should not deter companies from taking the first steps. The first step might well mean cleaning only some data.
“”Don’t spend money and time needlessly and don’t pursue quality for quality’s sake,”” he says. “”Not every table in your ERP is hurting you today. Focus on the critical data and work outward from there, and base the frequency of scrubbing on the volatility of data.””
The bottom line, he says, is that companies will see the ROI of a high IQ once they start the process, but there’s no reason to wait.
