Advantages and challenges of running a Random Probability Project

ESOMAR’s Global Prices Study 2023 report features the Random Probability Sampling project as one of the eight typical market research projects. In random probability sampling projects, agencies employ techniques designed to extract a representative sample from the population, thereby reducing the bias and making the findings more likely to be representative of the market. Andrew Cleary shares his views on the advantages and challenges of conducting a random probability sampling project in this extract from the report.

A random probability sample is the "gold" standard of survey research. The maths behind a random probability sample is that it satisfies two criteria: every unit in the population has a chance of being selected for the sample (i.e. the sample has ‘full coverage’ of the survey population), and the probability of selection for any unit in the population is either known or could be calculated. Effectively, everyone in the population has a known and non-zero chance of being selected. Random probability sample surveys can work with any data collection mode, providing the procedures are followed.

Probability samples should be used whenever point estimates (e.g., means and percentages) need to be generalised to the population and/or where accuracy is paramount.  Strictly speaking, only random probability samples can be generalised to a larger population within known limits (i.e., confidence intervals), although in practice, these ranges are often used as a guide to the likely margin of error of quota samples.

Random probability sample surveys are generally more difficult and costly to execute than other surveys. In all cases, it is necessary to identify and access a comprehensive sampling frame which lists all (or nearly all) members of the survey population or to build one from scratch. Specialist skills are needed at the sampling stage, particularly given the wide range of more complex designs available to tackle different situations. In the field, most random sample surveys are coupled with efforts to maximise the response rate to minimise non-response bias (i.e., improve sample accuracy). This requires more extensive procedures to contact the full sample (e.g., multiple contact attempts, varying the time/day of contact, fully working all sample cases to the procedures) and efforts to persuade individuals to take part, such as incentives and use of well-trained interviewers and/or persuasive materials. This adds up to greater costs and time to fully execute.

Andrew Cleary is the Head of International Research Methods at Ipsos

Are you curious about your agency’s prices compared to the median prices at global, regional, and local levels? Head to ESOMAR’s Global Prices Study 2023 to download the report and the data tool to benchmark your agency’s prices!

The report also includes relevant topics for the industry, such as the cost of a standalone F2F/virtual presentation, the choice of data collection methodologies, and the evolution of professional tariffs, along with opinions of industry experts. A must-read for both suppliers and buyers in the industry, available on the ESOMAR website.