How to Exorcise Your Data Demons

Image Source: Screen Capture from Host (2020)Happy Halloween! For millennia, demons have been a fixture in superstitions and religions for many cultures. More recently, they’ve found roles in modern films, starting with the 1973 classic The Exorcist. In the Bible, demons are malevolent tricksters who use people to carry out their evil intentions. In the Quran, djinn can drive people mad with illness and seizures. In Jewish mythology, the Mazzikim are the most insidious - the source of the stress and pressure you might encounter at work every day. And we know how haunting that can be.

People aren’t safe from possession, and our systems aren’t safe from it either… at least according to a whole genre of modern techno-horror films. In the Ring (1998), demons living in a VHS video tape can possess those who watch it. Pulse (2001) follows an internet service provider that gets its subscribers high-bandwidth connections to a whole host of troublesome spirits. In Host (2020), shown in the picture above, a Zoom-based séance exposes its participants to something even more deadly than COVID-19. Demons work by preying on the innocent, by making promises that are too good to be true, and other kinds of deception that are rarely recognized until it’s too late. 

Each of these examples shows demons using technology to find people at home. But what about all those systems you encounter at work? Turns out, you’re not alone there, either.

Today, demons hide in our data, our data management ecosystems, and our governance frameworks… sometimes in plain sight. What’s the difference between the demons of old, and the demons we unwittingly feed every day? And where can we find - and rid ourselves of - these damaging (and distressingly mundane) villains? Here are just a few places to start:

Demon #1 - Copies. It all starts very innocently: one person notices that the data they use to do their work is changing all the time. They need something that’s stable, so they make a copy they can control, and use that. Eventually, they leave the company, and take their institutional memory with them. As this pattern plays out over time (and in different departments), the end result is a tsunami of unmanageable replicas that can’t be vanquished because the business (somehow, some way!) depends on them.

Demon #2 - The Modern Data Stack. There are so many products on the market now that sound like they’ll solve all of your data management and data quality needs… instantaneously! But it’s rare for a product to deliver lofty promises, no matter how magical those promises sound. Additionally, each software product you introduce into your ecosystem expands the complexity of the communication that will be needed to maintain the collection of them. More products = more work, and more opportunities for human miscommunication and error.

Demon #3 - Data Debt. This arises because someone puts off doing things they really should have done in the first place, doesn’t take care of basic data hygiene, or doesn’t adhere to governance (the standards and practices that help people, processes, systems, and data work together smoothly and productively). This demon accomplishes its work through deception. Once, when I asked a mid-level manager what their plans were for addressing technical debt in their pipelines, I got this spine-chilling response: “It’s not technical debt if you never admit you’re wrong.”

The demons in your data prey on innocent, well-meaning people to feed their power. They take control of systems by getting control of people, and they take control of people by burrowing themselves into systems. They work through deception, and convince you that there’s no problem, or that “it’s really not that bad.”

So how do you exorcise these demons? In the movies, you just name them, then ask them to leave (usually in elaborate rituals involving tools like holy water) or force them to leave (by destroying the vessels in which they live). It’s sort of the same at work.

First, you have to name the demons and make everyone on your teams aware that they exist. (Next, you may have to convince people that these actually are demons that will haunt you later, not non-issues that require no attention now.) Next, you have to force those demons to leave (just asking won’t do a thing). Unfortunately, unlike in the movies, you’ll probably have to hire a dedicated team of full time, dedicated exorcists for this all-consuming task. Someone’s got to watch for demons… their numbers are legion, far more than the three cases described above.

Your exorcists are the architects who will spend their days designing problems out of your systems before those problems can emerge. They will gather your workforce together to systematically rid your organizations of the demons that poison your data and turn well-meaning people against one another. 

Ultranauts helps companies establish and continually improve data quality through efficient, effective data governance frameworks and other aspects of quality management for data and analytics, especially high impact data value audits. If you need to design quality into your ESG program, Ultranauts can quickly help you identify opportunities for improvement that will drive value, reduce costs, and increase impact.