The square peg in the round hole? Part 1

This article is part one of a two-part article series that covers challenges of business intelligence tools in handling market research data, and ways insights professionals can use fit-for-purpose technology to improve their ROI and workflow. 

Business intelligence tools have an undeniable place in the modern enterprise world - their ability to ingest, process, and report on a wide range of business data is second to none. The challenge for business leaders is to understand both their strengths and weaknesses. The strengths because often they have such wide-ranging capability that they are underutilised. Their weaknesses are often related to this said strength. People assume that they can handle pretty much any type of data, insisting on using the BI platform in situations where they aren’t a natural choice.

So, why do we continue to attempt to fit a square peg in a round hole when it comes to analysing market research data? Business intelligence (BI) tools, while they have their purpose and strengths, simply are not the best solution for insights professionals to get the most out of their primary research data. With more than 32 years in the market research industry, our team has worked with some of the globe’s largest brands and seen the frustrations that ensue from trying to make BI tools do work that is outside their primary function - major brands who have opted for some of the more well-known BI platforms, only to realise the technology’s limitations and subsequently pull the plug on the implementation. It’s now time to talk about it. 

Some people who use BI tools prescribed by their organisation’s IT department have gotten used to all the workarounds and shortcomings the tools have in processing complex primary research data. We’ve even come across some that were generally impressive. But one can only imagine the costs of bespoke development required to re-engineer the platform to process market research data in a way that respects its context. 

But, not everyone can afford such resource-heavy projects. And not everyone is confident enough to continually trust those workarounds, which are at risk of time delays. We need things faster, we need more holistic data and we need absolute accuracy. Here are five common challenges we’ve seen over the years for brands who are using BI tools for market research data: 

Data input. Different types of data require different types of processing. As people have found with data lakes, chucking all sorts of datasets into a lake can see it quickly become a swamp. As a result, domain context is lost, instantly stripping a level of value from the data. Many BI tools are designed for aggregate and relational data, but can’t process respondent-level data or multiple data inputs. Yes, significant workarounds can be made to accommodate these two types of market research data, but that doesn’t come cheap.    

Lack of specificity. In market research, we like to look at data in myriad ways. BI tools don’t easily allow you to accomplish tasks like examining weighted data, significance testing, processing multi-response questions, or building metrics on the fly, leaving you at a serious disadvantage. Again, workarounds can be made, but this will require either extra development or additional manual analysis.

Not shareable. When source data needs to be manually imported, and reports are not designed for direct client or stakeholder interaction, you run into major inefficiencies. These extra steps in producing outputs from data consume extra time and increase the potential for errors. If you can only produce static files, like PDFs or PowerPoints, then people aren’t viewing the most up-to-date insights. 

User-unfriendly. When you have to jump through hoops and create workarounds to do your job and deliver insights, it’s not ideal and, worse, it is very slow. Additionally, when only data experts with querying experience and data model knowledge can create meaningful analyses, many people are left out.

Under-automated. Automation is the only way to deliver the speed and accuracy the market demands. It can complete tasks - such as bringing together multiple data sources - in minutes rather than days without manual intervention. BI tools can fall short if researchers are forced to set endless crosstabs, and other time-consuming tasks. BI tools also typically fail to automatically highlight significant differences or similarities among audience segments. 

Tune in for the second part in this series to find out how to approach solving these key challenges, coming up next week.