2023-01-31, Kosch, Thomas, Karolus, Jakob, Zagermann, Johannes, Reiterer, Harald, Schmidt, Albrecht, Woźniak, Paweł W.

The ever-increasing number of computing devices around us results in more and more systems competing for our attention, making cognitive workload a crucial factor for the user experience of human-computer interfaces. Research in Human-Computer Interaction (HCI) has used various metrics to determine users’ mental demands. However, there needs to be a systematic way to choose an appropriate and effective measure for cognitive workload in experimental setups, posing a challenge to their reproducibility. We present a literature survey of past and current metrics for cognitive workload used throughout HCI literature to address this challenge. By initially exploring what cognitive workload resembles in the HCI context, we derive a categorization supporting researchers and practitioners in selecting cognitive workload metrics for system design and evaluation. We conclude with three following research gaps: (1) defining and interpreting cognitive workload in HCI, (2) the hidden cost of the NASA-TLX, and (3) HCI research as a catalyst for workload-aware systems, highlighting that HCI research has to deepen and conceptualize the understanding of cognitive workload in the context of interactive computing systems.

2022, Mueller, Florian 'Floyd', Semertzidis, Nathan, Andres, Josh, Weigel, Martin, Nanayakkara, Suranga, Patibanda, Rakesh, Li, Zhuying, Strohmeier, Paul, Knibbe, Jarrod, Greuter, Stefan, Obrist, Marianna, Maes, Pattie, Wang, Dakuo, Wolf, Katrin, Gerber, Liz, Marshall, Joe, Kunze, Kai, Grudin, Jonathan, Reiterer, Harald, Byrne, Richard

Human-Computer Integration (HInt) is an emerging new paradigm in the human-computer interaction (HCI) field. Its goal is to integrate the human body and the computational machine. This monograph presents two key dimensions of Human-Computer Integration (bodily agency and bodily ownership) and proposes a set of challenges that we believe need to be resolved in order to bring the paradigm forward. Ultimately, our work aims to facilitate a more structured investigation into human body and computational machine integration.

2021-01-27, Wang, Yunlong, König, Laura M., Reiterer, Harald

Prolonged sedentary behavior is related to a number of risk factors for chronic diseases. Given the high prevalence of sedentary behavior in daily life, simple yet practical solutions for behavior change are needed to avoid detrimental health effects.

2020-10-14, Ziesemer, Katrin, König, Laura M., Boushey, Carol Jo, Villinger, Karoline, Wahl, Deborah R., Butscher, Simon, Müller, Jens, Reiterer, Harald, Schupp, Harald T., Renner, Britta

Background: Establishing a methodology for assessing nutritional behavior comprehensively and accurately poses a great challenge. Mobile technologies such as mobile image-based food recording apps enable eating events to be assessed in the moment in real time, thereby reducing memory biases inherent in retrospective food records. However, users might find it challenging to take images of the food they consume at every eating event over an extended period, which might lead to incomplete records of eating events (missing events).

Objective: Analyzing data from 3 studies that used mobile image-based food recording apps and varied in their technical enrichment, this study aims to assess how often eating events (meals and snacks) were missed over a period of 8 days in a naturalistic setting by comparing the number of recorded events with the number of normative expected events, over time, and with recollections of missing events.

Methods: Participants in 3 event-based Ecological Momentary Assessment (EMA) studies using mobile image-based dietary assessments were asked to record all eating events (study 1, N=38, 1070 eating events; study 2, N=35, 934 eating events; study 3, N=110, 3469 eating events). Study 1 used a basic app; study 2 included 1 fixed reminder and the possibility to add meals after the actual eating events occurred instead of in the moment (addendum); and study 3 included 2 fixed reminders, an addendum feature, and the option to record skipped meals. The number of recalled missed events and their reasons were assessed by semistructured interviews after the EMA period (studies 1 and 2) and daily questionnaires (study 3).

Results: Overall, 183 participants reported 5473 eating events. Although the momentary adherence rate as indexed by a comparison with normative expected events was generally high across all 3 studies, a differential pattern of results emerged with a higher rate of logged meals in the more technically intensive study 3. Multilevel models for the logging trajectories of reported meals in all 3 studies showed a significant, albeit small, decline over time (b=−.11 to −.14, Ps<.001, pseudo-R²=0.04-0.06), mainly because of a drop in reported snacks between days 1 and 2. Intraclass coefficients indicated that 38% or less of the observed variance was because of individual differences. The most common reasons for missing events were competing activities and technical issues, whereas situational barriers were less important.

Conclusions: Three different indicators (normative, time stability, and recalled missing events) consistently indicated missing events. However, given the intensive nature of diet EMA protocols, the effect sizes were rather small and the logging trajectories over time were remarkably stable. Moreover, the individual’s actual state and context seemed to exert a greater influence on adherence rates than stable individual differences, which emphasizes the need for a more nuanced understanding of the factors that affect momentary adherence.

2022-10-19, Babic, Teo, Reiterer, Harald, Haller, Michael

Over decades, many researchers developed complex in-lab systems with the overall goal to track multiple body parts of the user for a richer and more powerful 2D/3D interaction with a distant display. In this work, we introduce a novel smartphone-based tracking approach that eliminates the need for complex tracking systems. Relying on simultaneous usage of the front and rear smartphone cameras, our solution enables rich spatial interactions with distant displays by combining touch input with hand-gesture input, body and head motion, as well as eye-gaze input. In this paper, we firstly present a taxonomy for classifying distant display interactions, providing an overview of enabling technologies, input modalities, and interaction techniques, spanning from 2D to 3D interactions. Further, we provide more details about our implementation—using off-the-shelf smartphones. Finally, we validate our system in a user study by a variety of 2D and 3D multimodal interaction techniques, including input refinement.

2022, Zagermann, Johannes, Hubenschmid, Sebastian, Balestrucci, Priscilla, Feuchtner, Tiare, Mayer, Sven, Ernst, Marc O., Schmidt, Albrecht, Reiterer, Harald

With increasing complexity in visual computing tasks, a single device may not be sufficient to adequately support the user’s workflow. Here, we can employ multi-device ecologies such as cross-device interaction, where a workflow can be split across multiple devices, each dedicated to a specific role. But what makes these multi-device ecologies compelling? Based on insights from our research, each device or interface component must contribute a complementary characteristic to increase the quality of interaction and further support users in their current activity. We establish the term complementary interfaces for such meaningful combinations of devices and modalities and provide an initial set of challenges. In addition, we demonstrate the value of complementarity with examples from within our own research.

2021, Renner, Britta, Breyer, Friedrich, Reiterer, Harald, Schupp, Harald T., Sonnentag, Sabine, Woll, Alexander

Einleitung: Das Ziel des Verbundprojektes SMARTACT ist es, Interventionen zu entwickeln und zu testen, mit denen das gesunde Ernährungsverhalten und die körperliche Aktivität unter Einsatz mobiler Technologie gefördert werden. Mobile Anwendungen bieten die Möglichkeit, effektive Interventionen in realen Kontext, d.h. „im Moment“ der Verhaltensausführung für größere Zielgruppen zur Verfügung zu stellen.
Methoden: Das interdisziplinäre Konsortium besteht aus zwei Themen- (SMARTFOOD, SMARTMOVE) und Methodenbereichen (SMARTMOBILITY, SMARTECONOMICS) sowie der Konstanzer Life-Studie. Die mobilen Interventionen, die technisch durch SMARTMOBILITY umgesetzt werden, basieren auf aktuellen Verhaltensmustern, Verhaltensauslösern und situativen Kontexten (Familie, Arbeitsplatz). Ein Hauptfokus liegt auf der Verbesserung der Teilnehmeraktivität durch Hinweisreize und der Aufrechterhaltung von Verhaltensänderungen durch Kurzinterventionen. Des Weiteren erfolgt eine Kosten-Nutzen-Analyse der verschiedenen Interventionen.
Ergebnisse: SMARTACT hat in den beiden Themenbereichen (SMARTFOOD, SMARTMOVE) und den verschiedenen Lebensbereichen (Arbeitsplatz und Familie) vielversprechende Ergebnisse gewonnen. Wichtige Beispiele beinhalten eine Meta-Analyse zur Effektivität mobiler Interventionen zu Änderungen der Ernährung und Gesundheitsmarkern, mobile Interventionen zur Steigerung der Zufriedenheit mit Essen und Änderungen der Ernährung anhand intuitiver Heuristiken und Zielsetzungsstrategien im Alltag sowie im Berufskontext und Familienverbund. Ferner wurden neue Konzepte zur Steigerung der körperlichen Aktivität entwickelt.
Fazit: Die Ergebnisse zeigen, dass mobile Interventionen im Bereich der Ernährung und körperlichen Aktivität einen wichtigen Beitrag zur Primärprävention leisten können.

2022-08-31, Chiossi, Francesco, Zagermann, Johannes, Karolus, Jakob, Rodrigues, Nils, Balestrucci, Priscilla, Weiskopf, Daniel, Ehinger, Benedikt, Feuchtner, Tiare, Reiterer, Harald, Chuang, Lewis L.

Adaptive visualization and interfaces pervade our everyday tasks to improve interaction from the point of view of user performance and experience. This approach allows using several user inputs, whether physiological, behavioral, qualitative, or multimodal combinations, to enhance the interaction. Due to the multitude of approaches, we outline the current research trends of inputs used to adapt visualizations and user interfaces. Moreover, we discuss methodological approaches used in mixed reality, physiological computing, visual analytics, and proficiency-aware systems. With this work, we provide an overview of the current research in adaptive systems.

2021-05-04, Mateescu, Magdalena, Pimmer, Christoph, Zahn, Carmen, Klinkhammer, Daniel, Reiterer, Harald

Large Interactive Displays (LIDs), such as tabletops or interactive walls, are promising innovations, which are increasingly used to support co-located collaboration. Yet the current evidence base on the impact of LID use on collaborative processes and outcomes, and associated influencing factors, is fragmented, particularly in comparison with other media. To address this gap, a systematic review was carried out in the databases Web of Science, Psych.Info, ACM, Elsevier, JSTOR and Springer and in the ACM CHI conference database. A corpus of 38 articles with experimental study designs met the eligibility criteria and was analyzed in-depth. With regard to collaboration processes, the findings suggest a relatively clear advantage of the use of LIDs over classic forms of collaboration, in particular over single-user environments (e.g. laptops). With attention to collaborative outcomes, positive effects of LIDs were identified for knowledge gains and social encounters, and mixed effects for task-related outcomes. The analysis further shows relevant influencing factors of LID, such as the separation of personal and joint work spaces and the deployment of horizontal instead of vertical displays. Conceptual and practice implications are discussed.

2020-11-11, Wunsch, Kathrin, Eckert, Tobias, Fiedler, Janis, Cleven, Laura, Niermann, Christina, Reiterer, Harald, Renner, Britta, Woll, Alexander

Background: Numerous smartphone apps are targeting physical activity and healthy eating, but empirical evidence on their effectiveness for initialization and maintenance of behavior change, especially in children and adolescents, is still limited.

Objective: The aim of this study was to conceptualize a theory-based and evidence-based mHealth intervention called SMARTFAMILY (SF) that targets physical activity and healthy eating in a collective family-based setting. Subsequently, the app will be refined and re-evaluated to analyze additional effects of just-in-time adaptive interventions (JITAIs) and gamification features.

Methods: A smartphone app based on behavior change theories and behavior change techniques was developed and implemented and will be evaluated with family members individually and cooperatively (SF trial). Existing evidence and gained results were used to refine and will be used to re-evaluate the app (SF2.0 trial). Both trials are cluster randomized controlled trials with 3 measurement occasions. The intervention group uses the app for 3 consecutive weeks, whereas the control group receives no treatment. Baseline measurements (T0) and postintervention measurements (T1) include physical activity (ie, self-reported and accelerometry) and healthy eating measurements (ie, self-reported fruit and vegetable intake) as the primary outcomes. The secondary outcomes (ie, self-reported) are intrinsic motivation, behavior-specific self-efficacy, and the family health climate, complemented by an intentional measure in SF2.0. Four weeks following T1, a follow-up assessment (T2) is completed by the participants, consisting of all questionnaire items to assess the stability of the intervention effects. Mixed-method analysis of covariance will be used to calculate the primary intervention effects (ie, physical activity, fruit and vegetable intake) while controlling for covariates, including family health climate, behavior-specific self-efficacy, and intrinsic motivation.

Results: This study is funded by the German Federal Ministry of Education and Research and ethically approved by the Karlsruhe Institute of Technology. For both trials, it is hypothesized that the apps will positively influence physical activity and healthy eating in the whole family. Furthermore, SF2.0 is expected to produce stronger effects (ie, higher effect sizes) compared to SF. SF app development and piloting are completed. Data acquisition for the SF trial is terminated and discontinued due to the COVID-19 pandemic. SF2.0 app development and piloting are completed, while data acquisition is ongoing. Participant recruitment for the SF 2.0 trial started in February 2020. The results for SF are expected to be published in mid-2021, and the results of SF2.0 are expected to be published in mid-2022.

Conclusions: In this study, it is hypothesized that targeting the whole family will facilitate behavior change at the individual level and the family level, as the implemented strategies address changes in daily family life. Furthermore, subsequent app development (SF2.0) with supplementary addition of motivation-enhancing features and a JITAI approach is expected to enhance positive intervention effects.