Brief non-spatial signals facilitate visual search and temporal sensitivity in robot supervision

Abstract

The human role in human–swarm interaction (HSI) shifts from controller to supervisor, as robots become more autonomous and require efficient search strategies in complex visual environments. Previous research has shown that spatially uninformative brief cues enhance search performance in laboratory environments (namely, “pip-and-pop” effect). Here we examined if these effects can be effectively applicable in HSI. To this end, we conducted two experiments using small mobile robots (Thymio II) to investigate the impact of auditory, tactile, and audiotactile cues on visual search performance and timing judgments. In the first experiment, 20 participants identified a stopped robot among moving robots. The results showed that all cue conditions significantly reduced reaction times (RTs) compared to the no-cue condition, suggesting that brief spatially non-informative signals improve search performance by increasing sensory information accumulation speed. The second experiment involved 12 participants judging the duration of a robot’s stop after a tactile cue was presented or not. The findings indicate that tactile cues improve temporal sensitivity without affecting subjective duration judgments. These results highlight the potential of uni- and multisensory cues to enhance HSI performance by facilitating quicker and more accurate human responses, particularly in dynamic environments. The study extends the “pip-and-pop” effect to real-world scenarios, offering insights for designing HSI systems that allow users to interact with robotic swarms more naturally and efficiently.