Survey quality and participant experience

The Insight250 spotlights and celebrates 250 of the world’s premier leaders and innovators in market research, consumer insights, and data-driven marketing. The inaugural list was revealed in April 2021, and the 2022 winners were announced in Toronto last September at ESOMAR’s Annual Congress. The awards have created renewed excitement across the industry while strengthening the connectivity of the market research community. The 2023 Winners were announced this past July and can be seen at Insight250.com. Also, the inaugural Insight250 Legends will be announced later this December.

With so many exceptional professionals named to the Insight250, it seems fitting to tap into their expertise and unique perspectives across various topics. This weekly series does just that: inquiring about the expert perspectives of many of these individuals in a series of short topical features. 

This edition features Paul Hudson, Chief Executive Officer for FlexMR. I sat with Paul to understand the threats and challenges to survey research and the tactics and best practices to elevate participant experience and research quality. His extensive expertise in strategic research provides a wealth of insight into how insights are becoming increasingly critical and, at the same time, increasingly challenging to deliver.

Today, industries have never had more research intelligence available. For example, you can access a spectrum of reports at mTab Marketplace from leading sources like Kantar, Tastewise, Brand Finance, Yelp, and Fatpos. These types of deep industry insights are helping brands understand their customers and develop their strategies like never before.

Crispin: There’s been lots of talk about survey quality and participant experience - what is this, and why is it important?

Survey quality covers a lot of elements, from fraud and bots to participant experience. It’s important for two reasons; one, because there is a ‘value-gap’ whereby participants don’t perceive they get the same value as organisations, and two, that leads to falling response rates and lost commercial value. If we can improve response rates, we’re all better off; we reduce costs, use time better, AND influence more decisions.”

Paul Hudson

Crispin: “What are the most common things that damage the participant experience?

PH: “Length and question structure, in my opinion, but it varies depending on segment/audience and what the survey is about. Work must be done to understand better what drives a better experience in different cases. Too much of this is based on the assumption that we know what is best.”

Paul Hudson

Crispin: What most undermines survey quality?

Commercial pressures. I know my team are very skilled researchers, but I also know they have many competing demands on them when writing a survey; they are balancing project objectives – the research question, time, cost, and competing client needs/demands. All of those influence what a survey finally looks like. And sometimes there isn’t a counterbalance in there about participant experience.”

Paul Hudson

Crispin: I hear you recently ran a Survey Quality Score (SQS) experiment - what’s that about?

Yep, that’s about creating that counter-balance, adding in the participants' experience as one of those competing demands, effectively giving participant experience ‘a seat at the table’ when we have those conversations about a survey and make design decisions. We need to be able to put the customer directly into that. That’s what the SQS is about: creating a measure of experience and benchmarking it across surveys, teams, and agencies, we can unlock some competitive spirit and motivate ourselves to improve. In my team, it is helping create a culture of ‘design-measure-improve’, and it will help moderate our client discussions when debating questions and length.”

Paul Hudson

Crispin: What did you discover? How do we improve survey quality?

At the moment, it’s early stages. We’re testing three things at present: 1) we’ve successfully tested the technical ability to collect scores from across different surveys in different agencies into one central database, 2) we’re now running research on research to ask customers what matters to them most in a good survey, and 3) we’re trialing different wording to how the SQS should work.”

Paul Hudson

Crispin:  At Congress, ESOMAR launched their Participant Experience Framework - tell me more about this.

That’s behind driving my work on the metric and benchmark. The ESOMAR framework is a guideline to help teams navigate what matters to participants. In our team, we’ve created a working group that is using that framework to review our work, challenge our survey designs across each element outlined and, ultimately, drive a focus on improving survey design.

Paul Hudson

Crispin:  How can more people get involved?

Firstly, the framework is already there. Secondly, people can join our evaluation project by adopting the SQS and building it into their surveys. If they use our API, we can all collect that data, build that benchmark, and evaluate its impact.

Paul Hudson

Crispin: If there were three things researchers should remember when designing questionnaires, what should they be?

Tough one! I’d probably go with length (fight that desire to add questions), language (balance the need to be unbiased/not leading with ease of understanding) and question choice (challenge your use of question type and match them to the audience and device).

Paul Hudson

Crispin: Great quality research and data is just the start. What helps get insights used? 

Absolutely. My other passion is driving Customer Salience for our clients. For me, the most important driver for getting insights used is to focus on ensuring we build cultures that always have the customer at the forefront of decision-maker minds; Challenge people to Create customer awareness, Connect them directly to their customers, and Collaborate on what the insights mean for their decisions.

“In 2022, Forrester reported that only 48% of decision-makers based their decisions on quantitative information and analysis, and according to Deloitte, only 37% place their companies at the top of their Insight Driven Maturity Scale, so there is a lot of work to be done!

“Our role needs to change from being researchers to becoming decision consultants and that’s a mind shift from collecting data to driving Customer Salience.”

Paul Hudson

Hot topics: AI

Crispin: You work at the intersection of technology and service to drive customer insight into every decision - how is AI helping?

AI can help us take more actions and decisions. Yes, it can speed our work up, but ultimately it can help improve the number of actions we can find and the quality of those decisions. For me, it’s as much about driving the volume and quality, not just cutting costs and making things faster or cheaper. For example, it can help improve analysis quality in open-ended and qualitative data, drive greater thought by probing qual answers, and summarise data succinctly, helping find trends. Yes, all of those things make things faster. Still, the result needs a higher number of better-quality decisions.”

Paul Hudson

Hot tips:

Keep learning and changing; stand still, and you will, ultimately, end up behind everyone else”

Paul Hudson

Crispin: Thanks, Paul. You've shared some important lessons and best practices as research becomes increasingly valuable to decision-making and strategy. Thank you for discussing your perspective and expertise on so many critical topics.