Data sets go unnoticed for a variety of reasons: a data set may be small in volume though not in value; it may be a large collection of data, overlooked because it relates to some activity not perceived as the main focus for the organisation. Whatever the reason, these pockets of data exist. While business-as-usual processes run smoothly, everything is assumed to be working as it should. No one thinks to turn out these pockets to see what they contain.

By some measures the assumption that everything is fine is perfectly valid: the application has a support contract; the data is backed up; there is no significant downtime. The challenge is that hidden data is both an unutilised resource and a governance risk.

In terms of utility, ‘traditional business reporting’ relies on being a summary of a complete picture. ‘Data science’ produces fresh insights when it overlays information from diverse data sets. Both techniques deliver better results when they are designed using a full understanding of the raw information they draw on. A conclusion drawn from data is more likely to be accurate if the information that goes into it is complete.

On the flip side, the risks associated with data that lie outside an organisation’s governance framework are enormous. At the very least, they undermine the quality of day-to-day decision-making. More seriously, they lay an organisation open to the kind of threats, bribery, money laundering, and fraud that cost executives their jobs and wipe vast sums from the share price.

A recurring theme – though far from the only one – is the political fault-lines that run through organisations. A typical scenario is the division of a company resulting from a merger. The barriers of differing cultures, languages, reporting lines, and personal relations impede the free movement of information.

The effect is no less significant than the more obvious barriers of time and money spent on integrating data systems. How often have systems been left behind because it was ‘legacy business’, the book was in ‘run off’, it was not included in the original estimate? When the integration project is over and everything is working, no one has reason to keep flagging these remaining systems as an open issue. Organisations have short memories because staff members change roles and documentation from past projects is seldom read.

That’s the problem: so what’s the solution?

KYD – Know Your Data

Organisations need to continuously maintain an accessible record of the data sets they hold. They need to know who is responsible, who is accountable, what each data set contains, and how it is accessed. They also need to know what information flows into a data set and where it goes when it’s taken out.

The maintenance of this record needs to be tied into the governance, risk, and control (GRC) fabric of the organisation, so that testing the controls provides assurance that the record is being maintained.

When the answer is so obvious it is almost surprising that it is not done consistently across major businesses. However, the reality is that when I am called on to performed data analytics for forensic investigations and customer remediations, I consistently find that these records do not exist.

What I can’t tell is whether the quality of data management is representative of all companies, or whether the companies whose records I get to see are somehow a self-selecting group.

Article: Part of our Data Leaders blog series.

Get in touch to discuss our services or this piece in more detail!