Paving the Road for Replications: Experimental Results from an Online Research Bibliography

RePEc Short ID: pre86
ORCID iD: 0000-0002-6459-8174

Coupé, Tom; Reed, W. Robert; Zimmermann, Christian

doi:10.20955/wp.2021.013

Working Paper Revision

Paving the Road for Replications: Experimental Results from an Online Research Bibliography

Abstract: Are users of a bibliographic database interested in learning about replications? Can they be induced to learn? To answer these questions, we performed an experiment at the online research bibliography, RePEc (Research Papers in Economics). RePEc is the main research bibliography for pre-prints and published papers in economics. Using randomized stratification, we allocated 324 replications and their corresponding original studies to clusters. We then drew from those clusters to construct treatment and control groups. Brightly colored tabs were added to the relevant webpages to alert visitors to the existence of a replication study. We then monitored traffic over three phases lasting several months: a) no treatment, b) treatment on one group, c) treatment on both groups. Our estimates indicate that this intervention generated an average click-through-rate (CTR) of 1.6%, resulting in a 13% increase in the visits to replication webpages, though only the former estimate was statistically significant.

Keywords: replications; RePEc; Experiment; Online Research Bibliography; Webpages; Click-throughs;

JEL Classification: A11; B41; Z0;

https://doi.org/10.20955/wp.2021.013