The reliability of questionnaires in laboratory experiments : What can we do?

Abstract

Questionnaires eliciting personality traits and other characteristics of a person are important tools for many experimental economists. While a lot is known about how to run experiments and about how to construct and run field surveys, much less is known about how to administer such surveys in a post-experimental context. A short survey among experimental economists documents substantial heterogeneity in the procedures used, and in expectations about the effects of procedural details. I run an experiment on five aspects that are specific to the laboratory context. I find that (i) paying participants as soon as they finish the questionnaire yields a lower answer quality than waiting for all or an intermediate procedure; (ii) having participants enter their names for receipt preparation does not reduce answer quality (and does not increase the social desirability of answers); (iii) a higher overall payment increases answer quality, while (iv) framing the fixed part of participants’ payment as being “for completing the questionnaire” as well as (v) progress feedback do not affect answer quality.