Usage of Online Panels in Survey Methodology Field: a Systematic Review

In addition to being a sample source for substantive research in social and marketing research, online panels are often used in survey methodology field. In particular, they serve as (1) an object of research itself or (2) a sample source for various experimental studies on survey data quality. This calls for a review of their usage and quality as used in survey methodology. Within this context we present a systematic review that aims to assess (i) the types of online panels which are used in survey methodology, (ii) the quality of these online panels, and (iii) the usage of online panels as a sample source for research on survey data quality. We use a bibliographic database by WebSM, which is the main information source on web survey methodology (Hewson et al., 2016; Callegaro et al., 2015; Lozar Manfreda and Vehovar, 2007), to select empirical studies from survey methodology that relate to online panels. To address our research questions we define specific coding categories for the type and other characteristics of online panels (e.g., panel size, target population, etc.), and purpose of the usage of online panels. Related to the purpose, we code various (i) panel data quality aspects for studies on the quality of the panel itself, and (ii) methodological questions for studies using panels as a sample source for survey methodology research. We analyzed 66 journal papers and book sections on survey methodology, published in English from 2012 to 2016, which refer to online panels. The results show that online panels are a valid sample source for various experimental studies in survey methodology. In addition, the systematic review highlights partially or entirely unexplored issues about the quality of panelists’ answers, while coverage and nonresponse issues in online panels are already well researched. This paper casts light on features characterising the setting of different types of online panels, points out the main indicators of their data quality, and studies the feasibility of the usage of online panels for survey methodology. A more general aim is to increase the knowledge on the usage of online panels, which is one of the current and future trends in survey research