Integrating AI into market research

Artificial Intelligence (AI) technologies are disrupting the insights industry. This necessitates an integrative framework for guiding companies through unprecedented digital transformation. 

AI can create artificial agents who use firm and consumer information to generate insights and suggest actions that are most likely to optimise market outcomes. To fully realise AI’s advantages while mitigating associated risks, industry leaders must establish an integrative framework with agreed-upon standards about how AI should be used for marketing and insights. 

Voluntary agreement among firms to industry standards is necessary to address several interrelated issues. These include updating traditional research methods and practices, deploying appropriate AI programs and technologies, overseeing business reorganisation, and addressing ethical concerns. Given the scope and complexity of these challenges, it’s clear that a comprehensive and unified approach is required to ensure the responsible use of AI in the market research and insights industry. 

AI Integration into market research  

To highlight why AI is becoming widely used in market research, consider that there were 4.95 billion active internet users in 2022. Each day, these users tweeted 650 million times, had over 300 million meetings on Zoom, posted 350 million photos to Facebook, and viewed 6,000 to 10,000 ads. 

In total, it is estimated that humans produced and consumed 94 zettabytes of data last year. To put it another way, this means about three million gigabytes of data were created and processed every second! Since AI is the best tool to explore this sea of information, market research companies are being compelled to adopt the technology. 

Even traditional research methods that have remained relatively resistant to change are being upgraded to reflect advances in AI. Using machine learning and specific algorithms, surveys can now be completely automated, focus groups and experimental studies can be conducted in the metaverse, and deep insights in qualitative and non-numerical data can be generated almost instantaneously. 

These developments significantly reduce the time and costs associated with market research projects. Previously laborious and expensive projects might now be completed in days or hours at a fraction of the price. However, since there are no widely accepted standards for incorporating AI into market research processes, relevant information on best practices for integrating this technology into the industry remains scattered and hard to access. Inefficiencies associated with AI adoption will only become more pronounced as market research companies begin to integrate increasingly complex AI technologies. 

Frameworks explaining the use of AI technologies in market research processes do exist, although they are often limited in scope. For instance, the Cross-Industry Standard Process for Data Mining (CRISP-DM) is a systematic, repeatable process that is commonly used for data mining and AI-driven project development. The framework is useful because it outlines a comprehensive series of steps that structure research processes that incorporate AI technology. Nevertheless, limitations of this and other existing frameworks can be observed when comparing their content. 

While frameworks like CRISP-DM provide information related to specific research operations, others clarify technical vagueness associated with the selection and integration of AI into overall business and research practices. For example, the Strategic Framework for Artificial Intelligence in Marketing highlights the importance of properly utilising AI capabilities. Rather than conceptualising AI as a ‘thinking machine’, it’s suggested that AI be viewed as an array of different intelligences – each of which can be used for the completion of different tasks. These intelligences are categorised into Mechanical, Thinking, and Feeling AI. Each has different abilities ranging in complexity, from the automation of routine and repetitive tasks to analysing human feelings and emotions to assess information related to consumers’ needs, preferences, and levels of satisfaction with services. 

Knowing how to properly integrate AI into research processes and under which conditions different types of AI should be deployed are just two of the challenges now facing market researchers. Unfortunately, due to fragmented literature, as well as the myriad of parochial and disconnected frameworks related to AI integration, staying informed and keeping up with the pace of change is likely to prove challenging for many.

The changing landscape of market research

It is estimated that marketing AI’s value will grow to over $100 billion by 2028. In a recent survey of market research decision-makers, 91% indicated that they view AI as an opportunity for the industry. Despite this, they are still well aware of the disruptive effects that AI will have. Most agree that many jobs will become redundant within the next five-to-ten years. These include major positions that are ubiquitous in the insights industry, including market research assistant, statistician, research analyst, data scientist, and market analyst. Half of those surveyed have already considered pursuing additional training, while almost a quarter is contemplating leaving the industry altogether. This suggests that most market research companies will be forced to reorganise their business practices, while research professionals will have to reevaluate their roles. 

In addition to technical and organisational challenges, the integration of AI technologies into market research is also calling attention to a number of emerging ethical issues. Some of these issues are constrained to specific technologies, such as algorithmic problems, while others can have widespread implications for society, such as privacy concerns. One of the biggest concerns surrounding the use of AI in market research is the threat of flawed algorithms perpetuating discriminatory practices in advertising due to inaccurate assumptions or predictions about certain individuals or groups. As such, broad guidance on AI auditing protocols is needed. Failure to address these issues might lead to serious legal and regulatory challenges, but it may also hurt the credibility and reputation of the industry in general. 

As AI solidifies its presence in the insights industry, it is becoming increasingly clear that a voluntary agreement among market research companies and practitioners is necessary to facilitate the adoption of these technologies. While AI can provide market researchers with valuable insights and help them improve their decision-making capabilities, the integration of these technologies poses a number of challenges. To ensure that AI is deployed ethically and responsibly, market research companies and practitioners require a comprehensive framework that establishes agreed-upon standards. Without this, the industry’s ability to realise the full benefits of AI will remain limited.