From history to practice: enhancing data quality in the final stages of market research

If you search for “data quality” in ESOMAR’s vast library dating back more than 75 years, you’ll find papers tackling this topic from as early as the 1970s, although the challenge has certainly been going on even longer than five decades. In 1972, the paper “Sources of Error in the Personal Interview” made a plea for reducing errors in survey interviews and focusing attention on raising fieldwork standards. Sound familiar? Looking back even further, Professor Richard Millar Devens coined the phrase “business intelligence” in 1875 to mean gathering information to gain competitive advantage. Is this where our data quality woes began?

While the conversation around data quality has ebbed and flowed over the years, it has never dropped off the radar completely. Although many fingers have been pointed, specific practices blamed, and band-aid solutions implemented, no one has been able to find the data quality magic wand. Over my years in the industry, I have uncovered one thing for certain: it is nearly impossible to overstate the importance of data quality in building true audience understanding.

Ensuring quality in data analysis and reporting

A typical market research project includes several stages, and focusing on data quality during each of these is, of course, critical. For example, after the business question is posed and the best approach for uncovering insights is determined, a research team might employ the following steps:

• Selecting sample

• Designing the survey questionnaire

• Sending the survey into the field

• Gathering the data

• Organise and analyse information and data

• Present findings

Here, I’m going to focus on the last two points. While the analysis and reporting stages are only as good as the data that is gathered, there are ways that you can continue to ensure quality throughout these final steps in the process. The right technology plays a large role in data quality at this point in the process, especially if it is purpose-built for market research data. Look for solutions that can:

Handle the data: In today's business landscape, the influx of data from various sources is commonplace - and on the rise. Effective solutions must have the capability to process diverse data streams and consolidate them for a comprehensive perspective. Find a solution that can bring everything together and is tailored to handle the complexities of market research data in particular so everything can be brought together, present a holistic view and become an easily updated “source of truth” for your business. Maintaining the integrity of the original data, no matter how it is used, analyzed and processed, is also an important factor when it comes to data quality.

Democratise the data: Insights don’t mean much if they aren’t shared with the right people in meaningful ways. In the past, many tools have limited access and analysis to data experts with querying experience and data model knowledge. Technology that can facilitate sharing data among stakeholders, requiring no specific expertise, can help elevate the insights function across organisations. By offering varying levels of access for individuals in different job functions - having bumpers in place, so to speak - data quality can be maintained, minimising data risk.

Visualise the data: Choosing a tool that goes beyond static presentations for data visualisation, such as interactive dashboards and real-time data updates, can help facilitate sharing and collaboration. This not only enhances the presentation of valuable insights but also allows users to challenge the status quo, experiment with new visualisations, and uncover hidden data patterns. With visually appealing, interactive, and dynamic reporting, you can simplify understanding, usage, and retention of insights, empowering informed decision-making and creating tangible value.

The longevity of discussions surrounding data quality underscores its perpetual importance in market research. Despite historical efforts and evolving practices, a universal solution to ensure data quality remains elusive, although we are poised to make progress with the new Global Data Quality Project - a cross-functional effort from associations around the world. As we navigate the final stages of analysis and reporting in market research, purpose-built technology, democratised data access, and advanced visualisation tools emerge as key elements to ensure and enhance data quality, empowering organisations to make informed decisions and derive tangible value from their data.