Working Paper

Modeling anchoring effects in sequential Likert scale questions

Abstract: Surveys in many different research fields rely on sequences of Likert scale questions to assess individuals' general attitudes toward a set of related topics. Most analyses of responses to such a series do not take into account the potential measurement error introduced by the context effect we dub \"sequential anchoring,\" which occurs when the rating for one question influences the rating given to the following question by favoring similar ratings. The presence of sequential anchoring can cause systematic bias in the study of relative ratings. We develop a latent-variable framework for question responses that capitalizes on different question orderings in the survey to identify the presence of sequential anchoring. We propose a parameter estimation algorithm and run simulations to test its effectiveness for different data-generating processes, sample sizes, and orderings. Finally, the model is applied to data in which eight payment instruments are rated on a five-point scale for each of six payment characteristics in the 2012 Survey of Consumer Payment Choice. We find consistent evidence of sequential anchoring, resulting in sizable differences in properties of relative ratings for certain instruments.

Keywords: survey bias; latent variable models; EM algorithm; Survey of Consumer Payment Choice;

JEL Classification: C83;

Bibliographic Information

Provider: Federal Reserve Bank of Boston

Part of Series: Working Papers

Publication Date: 2013-12-09

Number: 13-15

Pages: 42 pages