ADIA > News > News and media > Release of ESOMAR-GRBN joint guidelines on Online Sample Quality

Release of ESOMAR-GRBN joint guidelines on Online Sample Quality

Mar 25 2015

Press release

View the Online Sample Quality Guideline

The market and social research sector has published guidance for brands and marketers, providing advice and techniques for ensuring the integrity of sample quality used in online research, a growing global market worth nearly $10bn.

The Guideline for Online Sample Quality (GOSQ), published by ESOMAR and the Global Research Business Network (GRBN), provides best practice guidance to help users and providers understand how evolving approaches to online sampling can impact and potentially compromise data quality.

Online sampling accounts for more than a quarter (28%) of global market and social research revenues at US$9.7bn, more than telephone and face-to-face surveying combined. It is now the main research model in the world’s top ten research markets (except France and China).

Most online samples are derived from panels of people who have agreed to answer questionnaires online, usually in exchange for an incentive. Market, opinion and social researchers use the results to provide a fast reflection of consumer views and opinions.

However, the rapid growth of online research has been accompanied by concerns about the integrity of the resulting research data. These are due to the number of professional research participants who enter multiple surveys to secure incentives or money, inattentive or untruthful respondents, unrepresentative target groups, and the potential for duplicated respondents as research providers broaden their sources to expand sample sizes.

The proliferation of mobile devices is also posing challenges as well as opportunities for online researchers, most notably the potential to create bias. Between 20 and 30 percent of research respondents now respond using a smartphone or mobile device – they are typically younger, male, and more ethnically diverse than in conventional sampling. The limitations of mobile devices can also skew completion rates. The guidelines underline the need for transparency when reporting data in this area.

Another area of concern relates to how online sampling has evolved over time, from online panels to routers to exchanges. Newer techniques such as ‘river sampling’ – recruiting samples from real time adverts or offers – can make it harder to validate respondent identities and their relevance to the target population.

In addition, the trend towards recruiting respondents from schemes such as ‘frequent flyer’ or particular websites opens up the potential for duplicate participants. Blending together samples from several panel sources to create greater balance can also lead to individuals answering surveys more than once.

Finn Raben, Director General of ESOMAR, said:

“The rapid growth in online research presents real challenges for brands and marketers to assess the quality of their samples. This Guideline is designed to help data users to understand the issues affecting online sample quality and so they can make judgements with full confidence. It underlines the need for a transparent sampling process, knowledge of how the sample was selected, and specifically to ensure that
the same individual only answers a particular survey once.”

The Guideline is recommended reading for all stakeholders in the research process, from survey designers to data users. It addresses the key issues of debate impacting on data quality, including:

  • Research participant validation to ensure the respondent falls within the description of the research sample.
  • Survey fraud prevention to ensure that the same person cannot receive more incentives by completing a survey more than once.
  • Survey engagement to ensure that the respondent is paying sufficient attention to the questionnaire and understands the questions being asked.
  • Category exclusions to ensure that the sample does not include respondents who might bias the results.
  • Transparency of sampling to build confidence in the likely accuracy of data quality and representativeness of the target sample.

Andrew Cannon, Executive Director of the Global Research Business Network, added:

“These joint guidelines are an important new step in providing global guidance on research issues which are becoming increasingly global in nature. GRBN and ESOMAR will be working with industry experts to publish further guidelines in rapidly evolving fields such as social media and mobile research for the benefit of researchers businesses and researchers across the globe.”