Designing Fluid Interaction for Immersive Analytics using Hybrid User Interfaces in Mixed Reality

Abstract

Immersive mixed reality environments are gradually becoming part of our everyday life. Yet, their sole reliance on spatial interaction paradigms presents inherent limitations for their broader applicability. One proposition, which is the focus of this dissertation, is to form a hybrid user interface by complementing mixed reality head-worn devices with familiar everyday devices, such as smartphones, tablets, and desktop computers. Although the concept of hybrid user interfaces has persisted for decades, no coherent delineation has emerged, leading to a distinct lack of consistent terminologies and design principles. To take a step towards understanding the potential of hybrid user interfaces in general, this dissertation explores their potential for fluid interaction in immersive analytics. Hybrid interfaces are especially well-suited for immersive analytics, where the benefits of immersive 3D visualization must be balanced with the precision and familiarity of 2D interaction. Thus, this dissertation addressed the overarching research goal of: How can hybrid user interfaces support fluid interaction in immersive analytics workflows? The first part of this dissertation addresses the foundational gap in prior literature. It introduces the concept of complementary interfaces to generally describe meaningful device combinations. It then contextualizes the concept of hybrid user interfaces within the current research landscape, thereby extracting key dimensions for describing and classifying hybrid user interfaces. The second part presents the design and empirical evaluation of four hybrid user interface exemplars, each providing a unique perspective on the overarching research goal. The exemplars explore different design principles of hybrid user interfaces, such as enabling seamless transitions between devices, applying effective task allocation across heterogeneous environments, or creating novel interaction techniques enabled through such meaningful device combinations. The evaluation of each prototype contributes context-specific design insights and research implications that are synthesized into broader principles. Overall, this dissertation revisits and contextualizes the fragmented concept of hybrid user interfaces within the current research landscape, providing design guidelines for achieving fluid interaction. While immersive analytics provides a fitting use case, the resulting guidelines extend to a wider range of scenarios, thus providing a foundation for the future of hybrid user interfaces.