Using kindness to generate better quality data

In financial thinking, the three-legged stool grounded in three sources of income has long been touted as the surest path to financial security. Given the research community’s ongoing issues with plummeting completion rates and rising incentives costs, we, too, need a trustworthy system that never tips, wobbles, or waivers, our own version of a three-legged stool. Here’s how to build it. 

The first leg is quality, as detailed by ISO 20252

By following the guidelines of ISO 20252, we generate research results that are valid, reliable, and actionable. Whether you’re a buyer or provider of research, this standard offers a wealth of information about the key requirements and issues to address, from questionnaire design to hiring third-party vendors, including behaving transparently and consistently following defined processes. This standard helps everyone involved in insights and analytics work understand the criteria that are necessary to generate quality data, insights, conclusions, and recommendations.

The second leg is ethics, as detailed by ESOMAR and other national associations. 

National and global research associations such as ESOMAR, CRIC, IA, and MRS offer ethics guidance via their codes of conduct. They prepare locally and universally relevant guidelines that outline how researchers should behave when working with data and people involved in the research process, whether that means clients, participants, vendors, research users, or others. 

Ethical research is carried out “honestly and objectively without infringing on privacy or creating disadvantages for those whose data is used.” (ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics, p4, 2016). For example, ethical behaviours would include not harming people, minimising the collection of personal data, allowing people to withdraw their participation, ensuring findings are supported by the data, and being honest and objective. 

For example, each of these behaviours, which keeps participants informed about their standing in the research process, could be considered ethical. 

• Once someone does not meet targeting criteria, we tell them, “You do not meet our targeting criteria. Thank you, and goodbye." 

• At the end of a project, we say, “You have completed all the questions. Goodbye.” 

• When someone doesn’t answer a question, we say, “Fix all the errors.” 

• Prior to beginning the research, we inform people that they will not receive incentives if they do not meet our screening criteria.

Finally, the third leg is kindness. But who do we turn to for this guidance?

Largely ignored in both quality and ethics standards, kindness is very personal. In fact, it’s quite easy to conduct quality and ethical research without being kind. If you cringed at all when reading the examples of ethical behaviours, you’ll know that many of our standard research behaviours can be extremely unkind.

What does kindness look like?

• Kind language: Kind language treats participants as respected friends. It uses commonly understood colloquial words and phrases that aren’t necessarily grammatically perfect. It includes saying please and thank you, even if those words could lower completion rates. It means offering encouraging words at the beginning, middle, and end of data collection.

• Kind explanations: Being kind means explaining in real words why you’re asking such personal health, financial, or demographic questions, e.g., “We know this question might feel uncomfortable, but we need to understand real-life situations.” It means giving kind yet true reasons for screening people out, e.g., “We need to ensure we've heard from a wide range of people.”

• Kind incentives: Being kind means sharing research results after a study has been completed because there is no such thing as a study where all the results are proprietary. It means providing incentives to every person who begins a study even if you later decide they don’t meet your criteria.

• Kind research designs: Since we know people prefer them, kind questions almost always use midpoints, even though research-on-research might say we get better data without midpoints. Kind questions allow for non-binary gender answers even if you (ignorantly) think that’s not a real thing. Kind questions use multi-select demographics because people have complex lives with work, school, child care, and elder care. Kind research oversamples people who are marginalised even though it costs more to do so.

• Putting participants ahead of researchers: Kind research puts the needs of people ahead of the needs of researchers, analytics, and statisticians, e.g., using 5-point scales rather than 11-point scales. When you know people won’t like something, find a different way to do it.

We know that great research depends on following the carefully thought-out ethics and standards of quality guidelines that our industry has built over the last few decades. We are fortunate to have a fantastic foundation.

However, if we truly want to address plummeting completion rates and soaring incentive costs, it’s time to stop researching just how bad data quality is. Stop researching why people hate participating in research and why they give bad data. Stop trying to find a new solution for a problem when we already know what the best solution is. The solution is to start being kind. Truly, genuinely kind.

Want to learn more about the impact of kindness on your questionnaire results?

Annie Pettit hosts an online course titled ‘How to Design Questionnaires That People Want to Answer’, and it’s available to you on-demand.

This exclusive on-demand training offers you access to six hours of content split into three engaging sessions. Designed to enhance your skills at your own pace and convenience, you’ll be able to download resources, see the trainer’s contact information and earn a certificate upon course completion.