Measuring Political Knowledge in Web-Based Surveys : An Experimental Validation of Visual Versus Verbal Instruments

Abstract

Does the opportunity to deliver visual instead of verbal stimuli of political knowledge to respondents in web-based surveys make a difference in terms of data quality? For instance, does the presentation of visual knowledge items reduce cheating, that is, looking up the answer via the Web? And do visual and verbal stimuli capture the same underlying construct? To test whether the use of visuals to measure political knowledge effectively makes a difference, we administer a question form experiment in an online survey of the German Longitudinal Election Study. Respondents are randomly assigned to one of two question formats—visual or verbal—and are asked to solve a set of eight questions on political leaders and their offices. The instruments are validated based on nonparametric item response theory and analyses of response latency. While there is no clear evidence for cheating behavior under either of the conditions, both instruments form strong knowledge scales. Results from a regression analysis indicate that the scales provide measures of closely related but not identical concepts.