Most of us look at reports, dashboards, or Excel spreadsheets to help us make business decisions. Have you ever wondered: Where did each value come from? What happened to it along the way? Can we trust it, and if not, how can we make changes that will help us trust it more? If these are the kinds of questions that stimulate your curiosity, you might be a Data Quality Analyst.
Data quality improvement requires process change. First, it’s important to understand what the process is. How are individual data elements acquired, assembled, transformed, and used to generate new information and insights? Who is allowed to create, see, update, or delete data or information? What governance processes ensure that people adhere to technical standards and standard operating procedures? These are the questions Data Quality Analysts seek to answer.
In contrast, Data Analysts trust that the process is sound, and the data meets at least minimum quality standards. Their work focuses on drawing out the reasoning between the questions business users ask, assembling queries and accessing the right data to answer those questions, and then preparing delivery mechanisms like reports and dashboards. Here’s a head to head comparison of the two roles:
|
Data Analyst |
Data Quality Analyst |
|
|
Primary Goal |
Drives insights from data and communicates results to business users |
Continually improves data quality and data quality management systems so that data analysts have better sources to work with |
|
Secondary Goals |
Build reports and dashboards; creates, documents, and analyzes data models (incl. schemas); handle datasets in different formats; interpret findings; triage problems and errors that are related to data and business applications |
Document data flows and the stories about how data and information is generated; build data catalogs and data dictionaries; elucidate data quality dimensions and appropriate quality controls; help eliminate waste in data management ecosystems |
|
Tools used |
Primarily mathematical and statistical: SQL, R, Tableau, Power BI, SAS/SPSS, RapidMiner, and sometimes Python or Excel |
Primarily conceptual: flow charts, process mapping, value stream mapping, data lineage, maybe some Excel, SQL or R |
|
Does this role create charts, graphs, and tables to communicate with data? |
Yes |
No |
|
Does this role plan and perform statistical tests to extract meaning from data? |
Yes |
No |
|
Does this role perform data profiling to understand the characteristics of available data? |
Sometimes |
Yes |
|
Does this role write data pipelines? |
Sometimes, but usually No |
No |
|
Does this role identify appropriate tests for accuracy, completeness, consistency and other data quality dimensions? |
No |
Yes |
|
Does this role write tests for data pipelines or data assets, and produce reports on those tests? |
No |
No |
While Data Analysts work primarily on the business side, Data Quality Analysts are boundary spanners who focus on connections between the technical work that Data Engineers do, and the work done by Data Analysts and Data Scientists. While Data Engineers might make sure that data is fundamentally fit for use by someone, a Data Quality Analyst understands what is needed for that data to be fit for a specific purpose.
Ultranauts engagement teams working on projects in our Data Quality Engineering (DQE) practice have at least one Data Quality Analyst. These people are responsible for understanding and documenting data flows, and are supported by Data Quality Engineers who serve as technical interpreters, investigate pipelines of all kinds, and write tests and quality controls to monitor data quality.
Most of the time, it’s forensics work! Together, Data Quality Analysts and Data Quality Engineers piece together the story about how the data and information you need begins its life and ends up on your desktop or mobile device - and how to systematically improve data quality at your organization by using that story.
Ultranauts helps companies establish and continually improve data quality through efficient, effective data governance and quality management for data, specializing in high impact data value audits. If you need to design quality into your data management practices, Ultranauts can quickly help you identify opportunities for improvement to drive value, reduce costs, and increase impact.