Versatile Immersive Virtual and Augmented Tangible OR : Using VR, AR and Tangibles to Support Surgical Practice

Abstract

Immersive technologies such as virtual reality (VR) and augmented reality (AR), in combination with advanced image segmentation and visualization, have considerable potential to improve and support a surgeon’s work. We demonstrate a solution to help surgeons plan and perform surgeries and educate future medical staff using VR, AR, and tangibles. A VR planning tool improves spatial understanding of an individual’s anatomy, a tangible organ model allows for intuitive interaction, and AR gives contactless access to medical images in the operating room. Additionally, we present improvements regarding point cloud representations to provide detailed visual information to a remote expert and about the remote expert. Therefore, we give an exemplary setup showing how recent interaction techniques and modalities benefit an area that can positively change the life of patients.