Sensitive Questions in Surveys : Testing the Item Sum Technique (IST) to Tackle Social Desirability Bias

Abstract

Many surveys contain sensitive questions, for example, questions on sexuality, substance abuse, self-reported delinquency, or attitudes to foreigners. When answering such questions in survey interviews, many respondents do not respond truthfully and overreport positively connoted behaviors or attitudes (e.g., having voted in elections) and underreport negatively connoted ones (e.g., shoplifting). Also, many respondents refuse to answer at all, especially to questions on income. These phenomena are called social desirability bias and cause survey data to be not valid. We report the planning, design, process, difficulties, and results from a research project in which the item sum technique (IST) was experimentally tested against standard direct questioning (DQ) to tackle the problem of social desirability bias. The IST is a special questioning technique that, by adding random noise to the respondents’ answers, anonymizes the interview situation and therefore raises respondent protection. In our project, IST was compared to DQ for the income question and a question on self-reported alcohol consumption. A CATI (computer-assisted telephone interviewing) survey (N = 499) in Mainz, Germany, provides the empirical data. The key findings show that nonresponse to the income question is reduced considerably by IST. Regarding self-reported alcohol consumption, the slightly higher estimates in IST format (as compared to DQ) fail to be statistically significant for the whole sample. For male and for higher educated respondents, however, IST yields significantly higher estimates of self-reported alcohol consumption.