Most organizations struggle with incomplete, inaccurate and outdated contact data in their data warehouses, CRM, ERP, CDP, and other systems. Unless you live under a rock, you’ve experienced the frustration.
Marketers, data scientists, analysts, sales professionals, and business leaders are all too familiar with the problem and the time and money that’s wasted as a result.
At some point, the frustration reaches a level where something needs to be done. It’s time to get the IT team in a room and ask:
“How do we fix our dirty data problem?”
Fortunately, IT folks are geniuses. Testament to their brilliance is the technology, software, and data they manage daily. Somehow they keep on top of it all and keep everything running. Dirty data problems are just another challenge to tackle.
There’s a laundry list of things IT Pros consider when diving into the dirty data problem. Here are the top 3.
#1. Is the data properly elementized at the source?
For example, it’s hard to have any control over a 60-character “address” field. But it’s broken up into components like: unit number, street number, street name, street direction, city, province, and postal code, then there’s a lot that can be done to analyze and improve the data.
#2. Are proper input data validation rules in place?
For example, it’s easy to limit what data gets entered into a postal code field and restrict it to the format A1A 1A1. Without validation, all sorts of junk will end up in the wrong places.
#3. Are updates being logged and date-stamped?
As data is updated over time, it’s very helpful to know what last happened, and when. Otherwise, data from 10 years ago will look the same as data entered yesterday. Update stamping makes prioritization and segmentation possible.
If your technical team addresses these top 3 issues, your data will immediately be in the top 25% of what’s out there in the wild.
All these things can be done in-house – DIY style.
What comes next is where DIY ends. And it has nothing to do with the skills of your IT Team.
Your IT Team will point you to an outside expert because they don’t have the resource data needed to effectively solve the problem.
Your next task is to validate the data inside the various fields and fill-in data that’s missing. Some small parts of this work might be a candidate for a DIY project (for example, it’s relatively easy to validate the province codes in the province field), but at some point, all organizations turn to an expert partner.
Consider this example. Customer Sammy Segal’s postal street address “12 – 455 Beverly Street”. It looks pretty good, doesn’t it? But does Beverly Street exist? And is there a building #455 on Beverly? If so, is there an apartment #12? To answer these questions, you’ll need reference data.
It’s possible to buy a database of every address in Canada to compare yours to it. So long as you keep this information current, you’ll be able to answer the question as to whether this address exists. Easy, right? Not so fast.
Does Sammy Segal still live at 12 – 455 Beverly Street? That’s very important to know, but difficult to answer. And the address database you bought won’t help.
See the problem? It’s like the layers of an onion.
Now you need to check a mover database like Canada Post’s National Change of Address (NCOA), to see if there’s an update available. And even if there is, you need a way to know if it’s the most current address – or if he moved again.
Of course, it’s also possible that Sammy has passed, in which case all this work is futile. So we need to check our records against the Canadian Deceased Registry as well.
Do you see where this is going? It gets complicated, fast.
We’ve only talked about Sammy and his street address. There are also cities, postal codes, phone numbers, email addresses, and other fields to validate – some of which aren’t even populated.
Even then, you’re not done yet.
We want to know more about Sammy, so we better meet his needs. For example: what’s his ethnic background, does he own or rent his home, and how old is he? Answers to these and other questions rely on other huge and specialized reference databases.
Anything is possible.
You could invest the time, talent, and money to bring all these resources in-house (and essentially create another Cleanlist company), but is that really the best use of your time and resources?
For most organizations, the answer is clearly no.
Even the biggest brands choose to partner with an expert who has these resources and spend a few cents to have each record cleaned and maintained. Isn’t this better than re-inventing the wheel?
What Are the Next Steps?
Cleanlist offers free data assessments to qualified organizations. An assessment will show you the opportunity that data cleaning and enrichment offers for your database. Once the assessment is complete, the Cleanlist team will present a data maintenance proposal.
Cleanlist is Canada’s largest customer data company. We clean, enrich, and validate business and consumer data. We’re also experts in data-driven document composition and Canada’s largest provider of data for digital and offline marketing. To learn more, visit us at Cleanlist.ca
