The looming threat of insights internalisation

Information for this article was extracted from ESOMAR’s Global Users & Buyers of Insights 2023 and can be further expanded by making use of the online dashboard that comes with the report.

The insights function has enjoyed increased levels of accessibility over time. Progressively lower costs to carry out research have translated into a reasonable question from those who used to act exclusively as clients of full-service market research agencies: which elements of the insights function could I develop internally, and which ones should be externalised to specialised market research agencies?

This question lies at the heart of one of the most notable shifts within the industry in recent years and has allowed for the emergence of an entire industry dedicated to facilitating this transition. As a result, today, almost half of all research projects are conducted internally. This is facilitated by companies specialised in each step of the process: from the collection of data to its analysis, from the presentation of the results through straightforward visualisations to the guidance of the company’s decision-makers through prediction and advisory.

The challenges these insights teams face may sound much too familiar to those veterans in full-service agencies: having to deliver more with fewer resources, a heightened level of uncertainty from the impact of Artificial Intelligence (AI), ensuring that their research impacts the organisation and their decision-making processes, and the challenge of ensuring sufficient levels of data quality from a sample of increasingly elusive respondents and individuals.

While some full-service agencies may fear the total internalisation of the research function and their inevitable demise, the truth is that these users of research and buyers of software and tools have a very clear idea of just how much they can and should internalise. And they are equally aware of the role research agencies play and what they should take on.

While respondents declared how many projects are carried out internally versus externally, this does not inform of the potential cost they would have had if externally sourced. Temptations to claim that “half of all research is done internally” thus, should be complemented by the disclaimer “in number of projects” to ensure that audiences understand the base of the calculation. Internalisation is most likely to happen with small, quick, and straightforward projects.

This dynamic makes two things clear.

• The first one is that the relationship between internal insights teams and external insights providers should look less like a client-provider exchange and more like a partner-to-partner one.

• The second one is that now that a substantial portion of insights can be extracted internally, external agencies and researchers should step up and either understand the client’s reality to provide strategic guidance or differentiate themselves by providing a level of expertise and specialisation that cannot be sourced internally.

The level of internalisation remains quite balanced across regions, ranging from 45% in Europe to 52% in Latin America. Unfortunately, observations from either the Middle East or Africa proved insufficient to extract a reliable estimation of their level of internal versus external research projects.

Perhaps the most striking shift can be observed in Asia Pacific, which has seen the share of internal projects grow from 41% in 2021, the lowest of all regions, to 50% in the 12 months leading to this study.

Conversely, North America has seen its ratio of project sourcing shift towards increased externalisation, moving from 42% in 2021 to 50% in 2023, despite half of all respondents in 2021 claiming that the share of internal projects was growing.

Europe and Latin America, however, continue with the same level of internalisation as in 2021. This, too, contrasts with the apparent expectation that both regions were going to increase project internalisation – particularly in Latin America where this forecast was shared by 2/3 of all respondents.

Despite fears of relentless internalisation of the insights function by teams of analysts within those companies that used to be considered “clients”, evidence points towards a more gradual evolution. Not everything is, or can, be internalised, and neither the future will bring the demise of project-based agencies in favour of a fully decentralised profession. Perhaps not complete internalisation, but other threats abound in an industry navigating fundamental shifts brought by AI, lower barriers of entry, and the need for more strategic guidance.

Discover more about where research is done in the Global Users & Buyers of Insights study.