Understanding the next insights revolution

This article was published in full in ESOMAR’s Global Users & Buyers of Insights 2023 report. Find more information and uncover the reality of users and buyers of insights by downloading the report.

After working in insights and analytics for more than 20 years, I have seen the evolution of our industry. When I started in market research, most of the job was conducted by agencies with very high-quality standards. However, we spent so much time in agencies developing the perfect studies that we did not have too much time to influence the business. To solve this, we made a big revolution: outsourcing most of the job to external agencies so we could be focused on better understanding and influencing the business. The result was great at that moment, as we dramatically improved our business influence. And then, the second revolution came, driven by technology.

Now consumers are changing habits faster than ever before, and data is more accessible to everybody. More and more tools (research platforms, social listening, machine learning, AI, etc.) allow market research to be conducted easier, faster and at a lower cost. As a result, insights teams at global and local companies have adapted rapidly and dramatically by conducting around 50% of their market research internally since 2020. And according to the recently conducted User & Buyer Global Insight Study, this level of 50% continues to be the same in 2023 and is expected to remain in the future. This new revolution is making our industry more agile and efficient, but a few questions arise. Are we really becoming more strategic? Has our business influence increased? Will doing so much internal work prevent us from being more strategic?

The internalisation of research
After the pre-pandemic boom of increasing internal market research, companies are reaching a 50%-50% balance between in- and outsourcing. In general, companies is insourcing easy methodologies and outsourcing more complex studies where agencies are expected to provide more added value. The key driver behind internalisation is technology. When new methodologies, such as survey platforms, data integration, analytical tools, and social listening, become easier, faster, and of lower cost, companies rapidly adopt them. Now, most companies have internal teams dedicated to the use of these tools. Internalisation not only drives agility, but internal teams also reach deeper insights as we better understanding and care about their business.

However, internalisation requires companies to invest in their in-house teams to ensure data quality and actionability of the insights. Talent should be trained on new skills and should have clear career paths to ensure business continuity. In addition, talent diversity is another critical factor of internalisation. Different skills are needed to take advantage of the new methodologies and tools: data scientists, storytellers, research experts and business experts.

The internalisation journey is raising the bar for both our internal, as well as external teams. According to ESOMAR’s study, external agencies should focus on three key areas to provide more added value: understanding the client’s business (20%), acting as a consultant providing strategic analysis (16%) and providing expertise (15%). The preferred agencies are the ones that really care about the business, take the time to understand the client’s needs, use their expertise to offer a solution and ensure to answer the business need in a way that will allow them to drive action.

Building and developing strong internal insights and analytics teams is a key competitive advantage that only big international companies can afford. And even these companies will continue to use external companies to run more complex and strategic work. Developing the right balance of in- and outsourcing will allow organisations to develop insights and analytics in the most efficient way. However, our job does not stop here. We need to transform data into insights, insights into foresights and foresights into strategies and business plans. Only by having an appropriately diverse team in terms of skills and seniority will ensure influence on the business. The key risk for medium size organisations is to be lost in tactical work, driving agility and efficiency, but losing more strategic influence.

The future of market research
Tech disruption is changing the market research industry by driving agility as never before. However, the challenge continues to be the same: How to be more strategic? How to lead and inspire business growth? The top three needs of the industry continue to be the same: actionable insights, influencing the organisation and being more agile. Importantly, as new methodologies are allowing everybody to have access to consumer information and there is a proliferation of data sources, our key added value is: what we do with the data.

Transforming data into insights is very important, but it is not enough. We need teams to transform insights into foresights. It is critical to develop models that help us not only to explain the past but, more importantly, predict the future to create different scenarios and help the business to shape the future. We need to invest time in translating our insights/foresights into inspiring presentations to drive change in the organisation. We need to be agents of change.

To note, even when the survey indicates that the market research budget is expected to be the same or up next year, the industry indicates that limited resources are their key challenge. Why? Because business demands are increasing. In this context, the key challenge is to develop a strong ecosystem between the client and the agency with diverse talent that allows us to develop strong insights, foresights, and business recommendations to guide the strategic agenda of our business. Market research departments should drive the long-term plans of businesses.

It is a great time to be in market research: data, insights and consumer centricity have never been more important. Teams have evolved – and should continue to do so – from tactical, talking order departments to more strategic departments that guide the future growth of the company.