Report

Battery order effects on relative ratings in Likert scales

Abstract: Likert-scale batteries, sequences of questions with the same ordinal response choices, are often used in surveys to collect information about attitudes on a related set of topics. Analysis of such data often focuses on the study of relative ratings or the likelihood that one item is given a lower (or higher) rating than another item. This work studies how different orderings of the items within a battery and, in particular, the relative location of items affect relative rating distributions. We take advantage of data from the 2012?2014 Survey of Consumer Payment Surveys, in which item order in six Likert-scale batteries is varied among respondents. We find that ordering effects are real and consistent across years. The most prominent effect relating to relative locations of items is that the farther one item is placed after another item, the more likely that item is to have a lower rating.

Keywords: survey design; hierarchical models; ordinal responses;

JEL Classification: C83;

Bibliographic Information

Provider: Federal Reserve Bank of Boston

Part of Series: Research Data Report

Publication Date: 2017-07-24

Number: 17-2

Pages: 25 pages