Bridging Realities in a Heartbeat : How Integrating Heartbeat Signals Supports Collaboration in Mixed Reality

Abstract

Mixed Reality (MR) has great potential for remote collaboration, as remote users can be integrated with co-located users while facilitating similar richness in verbal and nonverbal communication, comprising gestures, facial expressions, voice, and body posture. However, current approaches mostly neglect more subtle cues, like physiological signals, which have been shown to be relevant during social interactions. We propose that physiological signals, such as heartbeats, should be integrated into virtual user representations to facilitate more effective remote collaboration in MR. In this work, we discuss the integration of heartbeats specifically, to enrich embodied interactions across the MR continuum, further bridging the gap between co-located and remote collaboration in MR. In addition, we identify key challenges and propose future research directions to support this vision.