## Complexity, learning effects, and plausibility of vignettes in factorial surveys

2009
Liebig, Stefan
##### Publication type
Working Paper/Technical Report
##### Abstract
The factorial survey is a research method of that combines the advantages of survey research with the advantages of experimental designs. Respondents react to hypothetical descriptions of objects or situations (vignettes) instead of answering single-item questions. By varying each dimension of the vignettes in an experimental design, the dimensions’ impact on respondents’ judgments or decisions can be estimated accurately. Thus, the method is able to isolate the weight of single factors which are often confounded in reality. So far, only a few methodological studies address questions of validity of measurement within factorial surveys. The paper provides an overview of the use of factorial surveys in social sciences and open methodological questions. Three of these methodological problems are discussed in more detail and studied empirically using data from an online-experiment: (1) the effects of different complex situations presented to respondents, (2) learning effects due to a repeated presentation of the vignettes and (3) the effects of implausible vignettes on respondents’ reactions. According to our results all three aspects matter: A high complexity of the vignettes leads to weaker effects of single dimensions while the consistency of the judgments remains the same. Until the tenth vignette there is evidence for a learning effect: Response speed increases at a stable rate of consistent responses. Implausible combinations of vignette dimensions cause the respondents to neglect the respective dimensions and therefore lead to artificial judgments. Finally, we discuss the practical consequences of these findings.
##### Subject (DDC)
300 Social Sciences, Sociology
##### Cite This
ISO 690AUSPURG, Katrin, Thomas HINZ, Stefan LIEBIG, 2009. Complexity, learning effects, and plausibility of vignettes in factorial surveys
BibTex
@techreport{Auspurg2009Compl-15080,
year={2009},
title={Complexity, learning effects, and plausibility of vignettes in factorial surveys},
author={Auspurg, Katrin and Hinz, Thomas and Liebig, Stefan},
note={Paper accepted for the ASA-Conference 2009. Working Paper # 4 of the DFG-Project "The Factorial Survey as a Method for Measuring Attituted in Population Surveys".}
}

RDF
<rdf:RDF
xmlns:dcterms="http://purl.org/dc/terms/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:bibo="http://purl.org/ontology/bibo/"
xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
xmlns:foaf="http://xmlns.com/foaf/0.1/"
xmlns:void="http://rdfs.org/ns/void#"
xmlns:xsd="http://www.w3.org/2001/XMLSchema#" >
<dc:creator>Liebig, Stefan</dc:creator>
<dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/34"/>
<dcterms:abstract xml:lang="eng">The factorial survey is a research method of that combines the advantages of survey research with the advantages of experimental designs. Respondents react to hypothetical descriptions of objects or situations (vignettes) instead of answering single-item questions. By varying each dimension of the vignettes in an experimental design, the dimensions’ impact on respondents’ judgments or decisions can be estimated accurately. Thus, the method is able to isolate the weight of single factors which are often confounded in reality. So far, only a few methodological studies address questions of validity of measurement within factorial surveys. The paper provides an overview of the use of factorial surveys in social sciences and open methodological questions. Three of these methodological problems are discussed in more detail and studied empirically using data from an online-experiment: (1) the effects of different complex situations presented to respondents, (2) learning effects due to a repeated presentation of the vignettes and (3) the effects of implausible vignettes on respondents’ reactions. According to our results all three aspects matter: A high complexity of the vignettes leads to weaker effects of single dimensions while the consistency of the judgments remains the same. Until the tenth vignette there is evidence for a learning effect: Response speed increases at a stable rate of consistent responses. Implausible combinations of vignette dimensions cause the respondents to neglect the respective dimensions and therefore lead to artificial judgments. Finally, we discuss the practical consequences of these findings.</dcterms:abstract>
<dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2011-12-07T14:17:51Z</dcterms:available>
<dc:contributor>Hinz, Thomas</dc:contributor>
<dcterms:issued>2009</dcterms:issued>
<dc:contributor>Auspurg, Katrin</dc:contributor>
<dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/15080/2/Hinz%20etal.pdf"/>
<dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
<dc:language>eng</dc:language>
<dc:rights>terms-of-use</dc:rights>
<dc:creator>Hinz, Thomas</dc:creator>
<bibo:uri rdf:resource="http://kops.uni-konstanz.de/handle/123456789/15080"/>
<dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2011-12-07T14:17:51Z</dc:date>
<foaf:homepage rdf:resource="http://localhost:8080/"/>
<dc:contributor>Liebig, Stefan</dc:contributor>
<dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/34"/>
<dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/15080/2/Hinz%20etal.pdf"/>
<void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
<dcterms:title>Complexity, learning effects, and plausibility of vignettes in factorial surveys</dcterms:title>
<dc:creator>Auspurg, Katrin</dc:creator>
</rdf:Description>
</rdf:RDF>

##### Comment on publication
Paper accepted for the ASA-Conference 2009. Working Paper # 4 of the DFG-Project "The Factorial Survey as a Method for Measuring Attituted in Population Surveys".
Yes