ARTHUR: authoring human–robot collaboration processes with augmented reality using hybrid user interfaces

Abstract

While augmented reality shows promise for supporting human–robot collaboration, creating such interactive systems still poses great challenges. Addressing this, we introduce ARTHUR, an open-source authoring tool for augmented reality-supported human–robot collaboration. We thereby propose to combine desktop interfaces and touchscreen devices for effective authoring with head-mounted displays for testing and in-situ refinements. ARTHUR supports 20 types of multi-modal feedback to convey robot, task, and system state, 10 actions that enable the user to control the robot and system, and 18 conditions for feedback customization and triggering of actions. By combining these elements, users can create interaction spaces, controls, and information visualizations in augmented reality for collaboration with robot arms. To demonstrate the general applicability of ARTHUR for human–robot collaboration scenarios, we replicate representative examples from prior work. Further, in an evaluation with five domain-savvy participants, we reflect on the usefulness of our hybrid user interface approach and the supported functionality, highlighting its potential and directions for future work.