Market Research is Dead - Long Live Insights

The End Game for MR

While the research industry dives headlong into AI, we thought it would be interesting to pose the question, what is the endgame here? If using AI is shifting the focus from doing research, from spending time getting data - and we are putting that time into insight generation and storytelling – surely, the ultimate outcome will be a total change of the role of the researcher?

We’ll be exploring this radical change in this article. Using the collaboration between the Worldwide Independent Network of Market Research (WIN) and AI automated analytics platform Inspirient, we will show how the role of the researcher can transition from traditional market research, to actionable insights, through AI.

Guaranteed, no hallucinations

Before we dive in, it’s important to differentiate the types of AI used in automated analytics engines. Large Language Models (LLMs), such as ChatGPT, are prevalent in market research and primarily rely on text-based inputs. In contrast, ‘walled-garden’ AI models, like the one used by Inspirient, are deterministically trained on curated sets of quantitative data. These models clean the data to eliminate biases and conduct analysis without relying on transcripts, videos, or imagery. 

Using a deterministic system, rather than a probabilistic LLM has several key advantages for researchers. You can control the data quality and create precise training parameters – you are not relying on the vast, and weird, internet. You get guaranteed data privacy for the company that has acquired the data – you are not putting your hugely expensive quant data out into the world. And, most importantly, you get guaranteed statistically correct results, with no made-up numbers - as you might find in LLM answers.

Global perspectives

The dataset used to train Inspirient in this project, was from the World Survey WIN has been conducting since 2018 which gathers people´s opinion on current topics, such as gender equality, health, climate change and sustainability. So, for the purposes of this project, Inspirient was ‘fed’ with real data from 39 countries, 25,000 interviews, multiple languages, and diverse contexts.

‘I have seen the future – and it works’

The collaboration between WIN and Inspirient aimed to assess how much time the platform saves for researchers. Findings revealed that, on average, AI saved human researchers at least 24 hours per typical project. 

The automated analytics engine revolutionizes workflows by streamlining traditionally time-consuming processes. Instead of navigating multiple manual steps, users can now concentrate on deeper analysis while the engine delivers key insights. For example, the WIN members in Peru redirected 12 hours toward developing a new analysis framework rather than preparing PowerPoint presentations. This shift from process-driven to insight-driven research enables a focus on strategic analysis, leading to more impactful outcomes.

Through this partnership, we have also found that traditional dashboards are not sufficient in delivering the impact and salience that data and insight need to drive decision-making. Interestingly, a key advantage here is to work around human biases in data evaluation and interpretation, in particular, the so-called “street-light effect”; a cognitive bias that favours searching for results that are easiest to find rather than the most valuable insights. Traditional dashboards can often also fall short in delivering the impact and relevance needed for effective decision-making, especially due to this ‘streetlight effect’. AI-driven automated analytics help overcome this bias by improving data interpretation and freeing researchers to focus on crafting compelling narratives and enhancing visual storytelling. For WIN, this was particularly helpful in communicating and interpreting across diverse cultural contexts, such as Peru, Vietnam, The Philippines, Ireland, and the UK.

Automated analytics engines offer researchers significant advantages beyond reducing a week-long analysis to minutes and minimizing biases - as demonstrated by the WIN and Inspirient case. A walled-garden AI engine is dependable for using credible sources, ensuring auditability with clear documentation, and uncovering insights that manual methods often overlook. 

As AI increasingly integrates into research processes, it is clear that these technologies will continue to augment the roles and tasks of researchers. Ongoing advancements in AI-driven analytics engines provide just a glimpse of their potential, signalling a transformative future for the insights industry.