dq:view - Steve Tuck on Data Quality

Customers: Corporate Asset or Corporate Liability

Financial services companies are risking their reputation and possible fiscal and custodial penalties by failing to recognise their exposure to potential criminal activity. As the deadline for implementation of the 3rd EU Money Laundering Directive fast approaches (15 December 2007) many money laundering reporting officers (MLROs) appear to be oblivious to the size of the problem they face.

The new directive further tightens the screw on financial services suppliers to know their customers. It requires them to take a 'risk-based' approach to screening their customers against prescribed sanctions lists and to also identify any client that is a politically exposed person (PEP). The legislation builds on existing efforts to prevent criminals access to the European Union’s financial systems.

It is up to individual firms to decide where they draw the line in the battle against organised crime and terrorists and there is little guidance from the Financial Services Authority (FSA), but any company that is subsequently judged to have acted negligently in this respect faces a heavy penalty.

I have to confess to a feeling of déjà vu. Following the Financial Services and Markets Act of 2000 and the Proceeds of Crime Act of 2002, Northern Bank and Bank of Scotland were each fined £1.25 million by the FSA for failing to take appropriate steps to prevent money laundering. Add to this the immeasurable damage these cases (and the headlines they attracted) did to the reputations of these organisations.

I have talked to many MLROs in a range of organisations, from small private banks to international asset management companies and retail banks with tens of millions of customers. Regardless of the size of the organisation, all of these people face the same challenge; deciding where to draw the line in customer screening to strike a balance between operational efficiency and crime prevention.

Attitudes range from the sublime to the ridiculous. Best practice is exemplified by the MLRO who measures every decision by what she describes as the Cornflake Test – doing everything necessary to ensure her employer does not appear in the morning paper for all the wrong reasons. This is stark contrast to the MLRO who suggested that doing nothing was a valid response to the risk-based approach required by the new directive.

Ineffective and uneconomical approaches

Anybody thinking of playing Russian Roulette with watch and PEP lists should bear in mind that World-Check, the leading supplier of consolidated lists, estimates that their file will contain approximately 500,000 names by the end of this year when the new directive comes into force. Financial services companies will need to regularly screen their entire customer base to ensure compliance.

Unsurprisingly, criminals do not like to be easily identified. So, despite the growing number of names on the lists supplied by the likes of Bank of England and the United States Office of Foreign Asset Control, they are becoming increasingly difficult to identify when hidden in a large corporate database. Whilst some resort to identity theft to cover their tracks, others simply manipulate their own names and personal details to create multiple personae.

Traditional approaches to matching customer names are proving ineffective and uneconomical when it comes to finding hidden criminals. They are typically unable to identify more complex matches, thereby missing the critical records.

These deficiencies have been evident in recent audits performed by my own company; each of them has found suspicious data that had been previously overlooked and each has resulted in new Suspicious Activity Reports (SARs) being filed by the company’s MLRO. The crimes involved have ranged from trading whilst insolvent to money laundering and even terrorism.

Meanwhile, some companies are loosening fuzzy match rules in an effort to perform a more thorough search. The main consequence of this is a large increase in the number of false positive matches produced, each of which takes time to review and approve. To make matters worse, compliance with the directive requires regular repeat screening of the entire customer base.

In the case of one large retail bank, this has resulted in the employment of 30 full-time personnel whose sole purpose is to review suspect matches. Whilst this is well intentioned, it is hugely inefficient and almost entirely unnecessary.

Thin end of the wedge

Manual review on such a large scale is a recipe for disaster. The operatives who perform such work soon become disheartened when it becomes apparent that they have to look at the same records time after time after time. This increases the chance of genuine matches to the sanctions lists being missed.

Because of the amount of manual review work involved, the bank is also unable to screen its complete customer list more than once per month and runs the risk of responding late to a new threat. This could be avoided by using a system capable of remembering and repeating previously made decisions so that only new or changed records required review.

Compliance with the 3rd EU Money Laundering Directive requires a blend of people, processes and technology:

• Money laundering reporting officer - all firms must have an MLRO, who must be sufficiently senior and be competent. The MLRO is responsible for internal and external reporting of exceptions.

• Training – the relevant staff must receive appropriate anti-money laundering training.

• Record keeping – details of suspicious individuals, organisations or activities must be reported.

• Customer screening – customers should be screened against the sanctions and PEP lists; not only at the inception of a relationship but on a regular basis.

• Suspect review – any possible matches should be reviewed in a timely manner; this may require referral to a client relationship manager. Once a review has been completed and the decision made, this does not need to be revisited unless circumstances change.

In today’s commercial environment, the damage caused by publicity following a breach of the EU directive could be devastating. Certainly, any fine imposed by the FSA is likely to be only the thin end of the wedge.

Third Party Data - Silver Bullet or Unreliable Evidence?

It can be very tempting to view third party data as a silver bullet to all your data quality woes.  take for instance the PAF (Postcode Address File), produced by the Royal Mail in the UK.  They describe it as "the most up-to-date and complete address database in the UK, containing over 27 million addresses."  I'm not going to take issue with their claim and I regularly recommend that organisations make use of PAF data. 

However, I think it's also worth pointing out a few things about PAF data:
1. PAF is produced by Royal Mail to aid with the effective and efficient delivery of postal items - therefore, it is only concerned with postal addresses, not all addressable locations (telecommunication and utility providers deliver services to many other addressable objects, such as streetlamps, traffic lights and road signs).
2. Contrary to popular belief, PAF does not contain a record for every business and residential unit in the UK.  Indeed, Royal Mail has actually removed some records for flats where they share a single mailbox.
3. PAF is updated regularly - but the changes can take months to be completely rolled out.  Many organisations fail to update their computer systems with PAF changes in a timely manner.
4. PAF is not infallible - it contains errors, omissions and duplicates.  Business addresses, particularly those  in business parks and those that rely on the business name are particularly prone to inaccuracies.

Authorative sources of data are indeed useful - just don't count on them to tell the truth, the whole truth and nothing but the truth.

Dell's hot technology

Dell has today announced the recall of more than 4 million laptop batteries over fears that they could overheat and start a fire.

.

There may well have been some data quality issues involved in the manufacture of these batteries (by Sony), but what concerns me is the opportunity for error when checking whether a battery is potentially dangerous or not.  Here's a shot of the label on my own laptop battery:

Can anybody tell me how many times the number 0 and the letter O appear in the code?

[click on the picture to expand it]

I was so uncertain that I tried a number of different permutations, with the following results:

Based on past experiences there is enormous potential for a significant number of these 4 million batteries to be left in circulation.  The website appears to offer no validation of the data entered whatsoever - you can type in anything and it won't complain.  This is no way to handle a safety recall, especially when the product labelling is so ambiguous.

For anyone that isn't squeamish, you can read a true story about how a laptop can get too hot to handle at The Register.

A Timely Reminder

To measure the quality of any data item we need to understand its definition.  Without that, we might totally misunderstand what we're looking at and if we're using the information as the basis for making an important decision the consequences can be dire.

Sometimes it's the presentation of the data that is at fault - take this example:

Date: 07/04/23

What is the date?  The 4th of July or the 7th of April?  And is the year 1923 or 2023?

If we were dealing with a customer database and the field was defined as the customer's date of birth I think we could safely assume that the year was 1923, but spot the same value in a field defined as a mortgage repayment date and the decision could go the other way.  As to resolving which is the day and which the month, we probably all jump one way or another based on what we're used to.  The problem is that the presentation of the date is ambiguous and without a clear definition it is open to misinterpretation.

Confusion over time zones can also cause fun!  I frequently have appointments for telephone calls with people in far away places and coordinating diaries can be problematic.  I know, for instance, that the east coast of North America is usually 5 hours behind UK time, but as we switch from EDT* to EST* and between BST* and GMT* we can be temporarily out of sync.  And why is it that we still change our clocks twice each year?  Is it just to confuse people more?

I was invited to a webinar by another data quality company today.  The invitation said that it would start at 2PM GMT - GMT, what madness is this?  We're in the middle of Summer so our clocks in the UK are all set to BST.  Did they mean that the webinar would start at 2PM, or did they mean it would be an hour later, which, if my brain is in gear, would be 2PM GMT?

It turns out that they were just using GMT as shorthand for UK Time and the event kicked off at 2PM on the dot.  It also finished promptly at 3PM BST - so woe betide anyone who was just logging in then for the scheduled 2PM GMT start!

I try to make a point of using the names of the months when I write out dates - particularly when I correspond with American friends and colleagues.  But is there an easy answer to solving the time zone issue - or should we all switch our clocks to UTC*?

* Glossary
BST - British Summer Time
EDT - Eastern Daylight Time
EST - Eastern Standard Time
GMT - Greenwich Mean Time
UTC - Coordinated Universal Time

It Can Happen to Even The Best of Us

Did you know that I'm the secretary of the IAIDQ's UK community of practise?  No, I thought not - and you probably don't know what the IAIDQ is either, eh?

The International Association for Information & Data Quality was established in 2004 to "cultivate information excellence" and "help transform organizations and society, improving the quality of life everywhere."  Blimey I think that's the first time I've read the vision statement - big or what?  I really look forward to seeing improvements in the quality of life EVERYWHERE!!!

My main objective in joining the association as a charter member (and then getting involved in some of its organisational challenges) was to facilitate a forum for people struggling to improve poor data quality.  The association has had very limited success in doing that to date (it's less than 2 years old, after all) and I have to admit to finding the name itself amusing (Information and Data are both included and I'll leave the anoraks to debate the difference between the two), but one thing that's clear is that the founders and executive of the association believe passionately that poor data quality costs organisations money (and sometimes worse).

Recently I received this from the association:

It's my new membership card and the thing to note is the dates; the valid through date is the same as the date I renewed my annual membership, i.e. it expired immediately.

The important thing though is that the association recognises that errors can occur and it provides a process to identify and correct them.  Beneath the signatures of founders Larry English and Tom Redman, is a request to "please examine the information below for accuracy" and telling us what to do if we find an error. 

In this case, the mistake was very speedily rectified and I received a new membership card.  Poor quality data cost the association some administration, printing and postal charges, but they acted quickly to deal with the problem and, I've no doubt, resolve the process issues that allowed it to happen.

I should close by saying one thing to the IAIDQ - thank you for your commitment to quality!

Data Quality is not to be sniffed at!

It was Fathers Day recently in the UK and my wonderful daughter presented me with a lovely gift of a personalised handkerchief.  Can you spot the deliberate mistake?  I'm sure she did it intentionally because of my nose for detail!

The comment in the bottom corner really tops things off -

Every time you blow [your nosie] think of me!

CDI - is it just another TLA (three letter acronym)?

I sometimes wonder if there’s substance behind the new catchphrases that the IT industry loves to throw about.  Take CDI, or Customer Data Integration, for example; is this some new, bright, shiny technology?  Or is it the cynical re-branding of something we already thought we had?

There is considerable confusion between CDI and that much-loved TLA (Three Letter Acronym) CRM – Customer Relationship Management.  CDI promises a single, 360 degree view of the customer; didn’t CRM promise the same thing?  I don’t blame anyone for being confused as, at it’s inception, I can remember myself thinking of CRM as an approach rather than a piece of technology and the single view of the customer was at the heart of it.  The truth is that, whilst CRM may have postulated the vision of a single customer view, the CRM vendors failed absolutely to deliver it.

That’s because CRM vendors focussed on the customer contact end of things; producing call-centre and sales-force automation technology, but did nothing to address the management of all the data that they collected.  Sure, the data model at the heart of any of these systems may support a single customer view, but without the processes to manage the quality of the data and identify and prevent duplicate customers from being created.

Furthermore, the processes that have been built around many of these systems have often encouraged the creation of dirty data rather than prevented it.  The most common name in one of my client’s CRM database was . .. (that’s a forename of “dot” and a surname of “dot dot”).  No, they had not started working in Morse Code, they had just paid the price for measuring the performance of their customer contact centre purely on call volumes, rather than the quality of service.

So what is Customer Data Integration and why is it different.  Firstly, let me say what it isn’t – it isn’t a piece of technology, the silver-bullet to solve all of your customer data challenges.  CDI should be a business objective – the management and coordination of all the information that a company holds on its customers.  For some, this may mean pulling all of their data into a single database and managing it there; but they will be the rare exceptions.

The reality is that most organisations will continue to live with multiple systems, including CRM, but will seek to create and manage the single view across all of them.  This presents them with two clear options: one where the data is federated and the single view created on the fly; and the other (I suspect more popular) option of creating a centralised “hub” that holds the master customer data with “spokes” that connects it to each of the other systems and synchronises data across them.

Critical to any of these approaches is the management of the quality of the data.  You can spend a fortune on a new car, but it won’t get you very far if you fill it with sludgy fuel and run it without a fuel filter.  In business, the challenge of dirty data must be tackled throughout the enterprise.

So is CDI something new?  Well I’m afraid, in true “there’s nothing new under the sun” style, the answer is no (I delivered my first CDI solution more than a decade ago, although the term was not widely in use then).  My hope is that the technology vendors supports the business community in achieving CDI as an objective, rather than uses it as just another way to sell CRM.

Gone Away - The need for Patient Data Integration in the NHS

In my blog entry on 28th March (Lost in migration), I discussed the issues surrounding the migration of patient data at the Nuffield Orthopaedic Centre.  This is part of a national programme to bring together patient data from different NHS Trusts to provide a "single patient view".

Each trust has been given a timetable to complete a CRS Migration, moving data from their Care Record Systems (CRS) or Patient Administration Systems (PAS) to a central "spine".  The Nuffield got a lot of press attention because it was one of the first trusts to complete a migration and ,as an out-patient at the centre I took a personal interest in it.

The need for Patient Data Integration is obvious; of course every clinician I see should have a complete history of my health treatment.  But just how disconnected the present system is was brought home to me again recently, courtesy of a good old "Gone Away".

My consultant at the Nuffield has decided to refer me to another department, which is based at the John Radcliffe Hospital (also in Oxford) and duly sent a note through to the appropriate person.  Of course, the easiest way of identifying me in the NHS computer systems is by my NHS Number, so this was supplied.  Unfortunately though, the systems at the Nuffield and the JRH are disconnected, so my details are stored separately on each.  And, having not received treatment at the JRH for some years, they still had my old address - so that's where they sent my appointment.

Thankfully, the current occupiers of the property still had a copy of our forwarding address and the letter arrived just a few days later.  The impact in this case was minimal, but consider what happens if a treatment centre cannot identify a casualty because their computer system is disconnected and out of date.  The diligence of the health workers means that most cases are resolved, but it takes time, effort and yes, money to do so.  Joining up the hundreds of NHS computer systems will improve efficiency, but most of all it will improve patient care - that's why NPfIT, the National Programme for IT, needs to succeed.

The Wrong Guy - identity crisis at the BBC

BBC New 24's coverage of the recent court case between the Beatles' record label, Apple Corps Ltd. and the computer firm, Apple Computer, Inc. was as comprehensive and professional as usual - until they suffered a case of mistaken identify!

The corporation had invited the journalist and author of newswireless.net to express his views on the verdict, but somehow, while the real Guy Kewney (pictured right) waited in the green room, they managed to get completely the "wrong guy" in front of the cameras - the look of terror on his face when he was introduced was priceless!...

But I have to say, the poor chap muddled through the interview in fine style; if you'd like to watch it, here it is!

Perhaps there's a clue to the reason he remained so unflappable in this gentleman's true identity.  At first it was reported that he was a London cabbie, at the BBC perhaps to collect Guy Kewney, but the truth is better than that: it turns out that the man featured in the interview is Guy Goma, a business Studies Graduate from the Congo who was in reception because he was applying for a high-level job at the BBC.  Apparantly Mr. Goma assumed that the whole thing was part of the recruitment process, but was "a little upset" that nobody asked him about his own area of expertise.

And what is Guy Coma's particular area of expertise? I'm pleased to reveal that it's Data Cleansing!  Well Guy, I hope you got the job in the end, I'm sure the BBC could use your experience, but failing that there might be an opening on "Working Lunch"!

dn:Director - a fresh approach to data quality

Why do so many organisations turn a blind-eye to data quality?  One thing for sure is that the legacy data quality software providers have done little to help address this crucial business issue by delivering products that require years of expertise to successfully leverage all of the functionality available (and, just as importantly, to know when to use something else instead).  After a dozen years of working in the field, and having built a highly profitable consultancy business to help clients address this short-fall, I decided a year or so ago to join Datanomic.  I'm delighted to say that, last month, we celebrated the launch of dn:Director, a data quality product that is setting new standards for data quality management in the 21st Century.

I've been privileged to work on data quality projects with many leading, blue-chip companies over the years, but one of the things that struck me was that I was being asked the same questions by clients in 2004 as I was asking myself more than a decade earlier; they were identifying the same old deficiencies in data quality products and having to employ the same workarounds to resolve them.  Sure, the vendors have done something to smarten up the look of their software, but, under the covers sits essentially the same code that was initially developed for mail-room efficiency in the 1980's.

Two more things struck me:

1. All of the software vendors talked about delivering a tool for "business users" but the reality was that just about every project relied on the IT department to develop the business rules.
2. Because of the complexity of using the software to good effect, the cost and duration of projects was prohibitive; the reason I was working with so many blue-chip companies was that they were the only ones that could afford to undertake such major projects!

These were the things that motivated me to create Tranato and subsequently to join Datanomic in 2005 and bring together the two technologies under a shared approach.  Put simply, we feel that a data quality product needs to be much more accessible - you shouldn't need to be a software guru to get value from it.

dn:Director is the result of many years experience in data quality and data management; not just my own, but that of people like Gerry Kelley (Datanomic's VP of Professional Services) and his team, and the shared experiences of our clients and partners.  Taking Datanomic's approach (The Four Cornerstones) and methodology as its foundations, dn:Director has been built from the ground up, using the best-available modern technology.

Developing dn:Director in Java and using standards-based interfaces (such as JDBC, JMS and XML) has enabled us to deliver a technically advanced and extensible data quality product that supports both batch and real-time processes (providing data quality services through SOA).  But the thing that everybody notices first is just how easy it is to use - you should hear what out customers and partners have had to say about it:

"This is great - it's so easy understand and configure business rules"

"I love the way that you can build rules from the data - it's so quick and intuitive"

"This will halve the time it takes to deliver a project"

For more information visit Datanomic's website or call on +44 (0)1223 228400.

Note: I know this is very commercial for a blog entry, but given the amount of personal time, energy (and money) I've committed to making dn:Director a success, I hope you'll forgive me.