Discussing data quality

The Insight250 spotlights and celebrates 250 of the world’s premier leaders and innovators in market research, consumer insights, and data-driven marketing. The inaugural list was revealed in April 2021, and the 2022 winners were announced in Toronto last September at ESOMAR’s Annual Congress. The awards have created renewed excitement across the industry whilst strengthening the connectivity of the market research community. The 2023 Winners were announced this past July and can be seen at Insight250.com.

With so many exceptional professionals named to the Insight250, it seems fitting to tap into their expertise and unique perspectives across various topics. This weekly series does just that: inquiring about the expert perspectives of many of these individuals in a series of short topical features. 

This edition features Debrah Harding, Managing Director of the Market Research Society. Debrah is also an Insight250 honoree. I sat down with Debrah to discuss various topics on the trends, advancements, evolution, and importance of data quality to market research. Debrah explains how data collection, use, and applications are evolving across a series of dimensions and how the importance of data is growing.

Crispin: Why is such an emphasis placed on data quality by the market and social research, consumer insights and analytics sectors?

Maintaining a high data standard is vital and something our sector works incredibly hard on. The actionable insights we produce that shape policy, allocate resources, and direct corporate strategies are led by our research. If the data is flawed, then the insights will be as well.

“This is much more than just ensuring the initial collection is representative. The questions should be unbiased and engage participants; equally, the data must be examined and possibly even weighted depending on the subject matter. It takes time and resources, but it is pivotal to our sector’s success.

“In the UK, we’ve been doing very well at keeping our data quality high. The emphasis on the UK research sector places on rigour and standards, and its reputation for doing so, is one of the reasons why we’re such a global hub and leader in this field. However, in an online world with bad actors and disruptive technologies, keeping the checks and balances in place that are required to maintain data quality is becoming increasingly challenging and costly.”

Why is data quality particularly topical now? Are new technologies such as artificial intelligence having an impact?

Many of the challenges research is currently facing regarding data quality remain similar to before. The sector is aware of the issues and invests a lot of resources to ensure standards stay high.

“While fraudsters looking to influence research for their own ends intentionally have made the headlines, poor participant experience is just as significant. As a sector, we must continue to challenge ourselves regarding research design. We must ensure participants fully engage, considering each question and responding thoughtfully rather than rushing through them for a reward.

“The movement to online-first research and wider technological advances have certainly provided new avenues for bad actors. There is also some evidence that disengaged participants are using ChatGPT to fill in surveys, for instance, while bots, artificial intelligence, and algorithms, despite having a positive role to play in research, can all be classed as weapons in fraudsters’ armouries.

“The sector is well aware of these threats and putting in checks to counter them. We have a variety of means to account for bad actors. The challenge is that this activity is taking up an increasing amount of time and money, putting further pressure on research teams already in high demand.”

How is MRS supporting efforts around data quality?

We run training courses to educate on research design and data quality and we host high-level forums where new techniques can be discussed and debated, including, of course, our annual conference. Equally, we lobby the government on behalf of the market and social research sector to ensure new laws such as the Data Protection and Digital Information Bill support and don’t hinder our efforts.

“However, all this isn’t enough. In a globalised world, we need international cooperation to help alleviate the pressure on research suppliers and to foster an environment where data quality can remain high. That’s why associations such as MRS, ESOMAR, and the Insights Association in the USA set up the Global Data Quality (GDQ) initiative with partners worldwide.

“Through this initiative, associations are working together on data quality, providing a forum to discuss the issues, and developing tools and valuable resources to support practitioners responsible for data quality. It means rather than tripping over one another and doing the same thing. Each GDQ partner owns part of the puzzle and is diving into the depths of a particular issue.

“MRS, for instance, is focused on fraud and bot technology, and a multi-disciplinary working group has already been created to manage this project. It will lead to a glossary of terms and definitions about bot and fraud technologies, as well as new solutions, approaches, and resources for researchers across the globe to use.

“It’s important that we’re not too inward-looking when it comes to this, and we’re already examining how other sectors are dealing with similar challenges. For example, the gambling industry and the financial sector are both heavily invested in data and working hard to combat bad actors influencing their work. If we can learn anything from them, we’re keen to do so; if they can learn anything from us, we’re keen to help.”

There have been data quality initiatives in the past. What makes this one different?

The scale of it and how seriously it is being taken. Such is the importance of the issue that associations from across the globe are working together, at pace, on this initiative. The sheer number of practitioners and businesses involved shows the commitment to tackling issues around data quality and developing tools and standards to help the sector improve going forward.

“Within the GDQ initiative, there is a lot of cooperation, and I’m sure that it will deepen international ties, which is really important in the unstable geopolitical climate we are experiencing today. Even at the early stage of this initiative, we’ve already seen our work on bot technology crossover with other projects led by the Austrian and American research associations, and we’re working closely with them so that we’re aligned.

“We do need to be realistic, though. Data quality is an evolving issue researchers have always had to engage with, and we will never solve it. The key is consistency, and that we establish principles and standards that the sector can work to. Significantly no end date has been set for this initiative. It needs to be ongoing and evolve as the nature of the challenges change.

“Nor is membership closed, and I know of several other associations who wish to contribute to the initiative. I would strongly encourage them to support it, be that in leading a workstream, providing a sounding board for the new tools being developed, communicating about the initiative, and encouraging their memberships to adopt the new tools and standards being developed.”

When can we expect to see results from this project?

We’re already seeing the first results from ESOMAR’s draft framework to improve research participants’ experiences and enhanced data quality.

“As we head into the Autumn, even more new materials will be published, including a new MRS and Insights Association glossary of terms and definitions for data quality that the international research community agrees upon and can be adopted when describing the challenges. It’s really important that we establish this early on. In the past, there have been differences in terminology, which has hindered understanding of the root causes and issues. Agreeing on them now will save time and create the building blocks for the more specific elements of the initiative.”

Do clients have a role to play in terms of data quality? 

Definitely. As part of the initiative, the Canadian Research Association (CRIC) is developing a guide that focuses on the questions that clients need to ask. It’s important that clients understand the challenges and how the sector is combating them.

“More than anything, though, clients should be making sure that they are working with reputable organisations that are members of their respective professional association, such as MRS. With this comes a guarantee of standards and quality that you cannot get in any other way.”

Considering the strains you’ve highlighted, how would you advise agencies, research teams, and individual practitioners to approach data quality now?

To engage with the GDQ initiative. Read, listen, share, and react to the content we produce – it’s all going up on the initiative’s website. But most importantly, make use of the work. Once new data quality best practices and standards have been developed, they must be adopted widely for the differences to be realised. 

“The more open we can all be on this issue, the better. By collaborating and sharing knowledge, the entire sector can improve and be more efficient.”

HOT TOPICS: Sustainability and Diversity & Inclusion

Two hot topics in our profession are sustainability, diversity, and inclusion. What is MRS doing to champion these, and what more should our profession do? 

At the heart of MRS’ work on inclusivity and sustainability are the MRS Inclusion and MRS Net Zero Pledges. These are a set of principles that MRS wants all CEOs in our sector to commit to.

“The MRS Inclusion Pledge already has over 100 signatories, all of whom have committed to ensuring there is no pay discrimination and that businesses are representative. As researchers looking to understand the motivations and the actions of the populace, it’s vital that we reflect them – otherwise, we may well find it challenging to engage with minority groups.

“Meanwhile, the MRS Net Zero Pledge asks companies to make four commitments including to be net zero carbon by 2026. I’m proud of how much work the sector is doing regarding sustainability and the insight we have provided on promoting more environmentally conscious behaviour. Again, it’s important that we reflect this in our own actions.

“If you are a CEO of a research business and haven’t yet signed up to one or both of the MRS pledges, I strongly encourage you to do so. If you’re a member of the wider sector, do get in touch, as there are plenty of other ways you can help our industry improve in these areas.”

Top Tip

We have many leading experts working on this, but we don’t know everything. If you have insight and advice to share, please do get in touch. We can then factor your experience into our thinking – we want to hear from you.”

Crispin: Thanks, Debrah - these are such important topics, and I really appreciate you sharing your perspectives AND all the hard work that MRS and other associations do in keeping them front of mind in our profession.