Assessing academic emotions via experience sampling methods

Abstract

One of the primary goals in research on academic emotions is to predict real-life emotional experiences in students and teachers. However, although prominent theories of emotions in achievement settings, such as Pekrun’s (2006) control-value theory, typically focus on intraindividual functioning (within-person relations), empirical studies in this area have primarily assessed interindividual (between-person) relations. In fact, Voelkle, Brose, Schmiedek, and Lindenberger (2014) suggest that ~90% of research in psychology entails the analysis of between-person variation, despite inter- and intraindividual relations being statistically independent (Molenaar & Campbell, 2009), calling into question the validity of typical inferences from interindividual findings to intraindividual functioning. This presentation outlines the experience sampling method (ESM) as an appropriate and sophisticated method for assessing intraindividual variability in emotional experiences that, although has been often used in occupational and health psychology research, has to date been underutilized in research on academic emotions (e.g., Bieg, Goetz, & Lipnevich, 2014). Following an explication of what experience sampling methods entail, as per seminal work by Hektner, Schmidt, and Csikszentmihalyi (2007), corresponding data collection protocols will be presented with respect to both electronic devices and paper-and-pencil diaries. Strengths of ESM will subsequently be addressed, such as the possibility to assess highly valid real-time data for use in intraindividual analyses, as well as the weaknesses of experience sampling, such as its time-intensive nature and treatment effects. The types of research questions well-suited to ESM will then be discussed as per potential analyses of within-individual fluctuations in academic emotions, with examples of research questions that are not well-suited to ESM also provided (e.g., real-time affect predicting decision-making). Given the challenges of analyzing ESM data, we further outline useful statistical methods specific to ESM protocols with respect to multi-level analyses, time-series analyses, and structural equation modeling. Practical advice for conducting ESM research on academic emotions will subsequently be provided in terms of recommended measures (e.g., Gogol et al., 2014), hardware and software options (e.g., iDialog Pad; Kubiak & Krog, 2012), and the appropriate number of assessments for specific research questions. Examples of ESM studies on academic emotions will also be provided, such a recent study by the authors showing girls to report more math anxiety than boys only on generalized self-report questionnaires and not during real-life math classes as assessed using ESM protocols (Goetz, Bieg, Luedtke, Pekrun, & Hall, 2013). The presentation will then close by outlining potential directions for future research utilizing ESM to assess emotions in academic settings, such as combining real-time self-reports with video data and/or biological markers. In summary, this presentation aims to inform and motivate researchers as to use ESM for the assessment of real-time academic emotions. This aim is particularly important given the potential for these methods to provide valuable intraindividual data to more appropriately evaluate relevant theoretical assertions with respect to intraindividual functioning. Moreover, this information is critical given the potential to also better inform the development of intervention programs and emotionally optimal learning environments to facilitate adjustment, learning, and performance in students and teachers alike.