PrIME: Primitive Interaction Tasks for Multi-Display Environments

Abstract

Multiple displays are commonly used in meetings and discussion rooms.<br />These settings provide new challenges for designing fluid interaction<br />across displays. For instance, a list of primitive interaction tasks in<br />such environments has not been investigated in literature. In addition,<br />past researchers have carried out their research using mobile phones as<br />input devices to control displays. However, it is still unclear whether an<br />input device with an integrated display can improve the performance of<br />multi-display interaction tasks.<br /><br />This research contributes to the design space of the multi-display<br />environments (MDE) in two aspects. Its first contribution is theoretical, as<br />it lists the primitive interaction tasks for MDEs. The indicated primitive<br />tasks are: object selection, object transfer and focusing-brushing-linking<br />in collaborative MDE, as well as visualization gallery as an extension to<br />the list of primitive tasks for single-display interaction. This list helps<br />interaction designers, who design new input devices, to be aware of the<br />tasks that the device should support. Furthermore, it can be used as<br />criteria to evaluate novel input devices.<br /><br /><br />The second contribution of this research is practical and it is aimed at<br />answering the following research questions, which the state-of-the-art<br />techniques have left unanswered.<br /><br /><br />Does a mobile input device with an integrated display improve<br />performing cross-display interaction tasks?<br />The idea behind this research was to compare two devices, one of which<br />has an integrated display whereas the other does not. To answer the<br />research question, a working prototype, called PrIME prototype, was<br />implemented. This was done using a laser-pointing device and an iPhoneas two alternative input devices for MDEs.<br />A user study was conducted to compare these devices according to their<br />performance, as well as for the users’ subjective feedback. The outcome<br />of the experiment indicated that the iPhone is better suited at selecting<br />overlapping objects, objects that are small, and objects which are at a<br />distance. Although it was hypothesized that the laser-pointing device<br />might be quicker than the iPhone to select larger objects, the paired<br />sampling t-test did not prove any significant difference between the two.<br />Transferring one object from one display to another was significantly<br />quicker using the iPhone. Surprisingly, the iPhone was not quicker at<br />transferring more objects from one display to two other displays. In fact,<br />the result of the t-test showed no significant difference. It was expected<br />that the iPhone would be quicker in performing more complex tasks,<br />namely where several tasks are done one after another. This was assumed<br />given that the iPhone has a clipboard which can save the selected objects<br />and which the user can carry around.<br />The users’ subjective feedback showed that the iPhone was considered<br />significantly better than the laser-pointing device. This is because it was<br />regarded as easier to use, more accurate for object selection, less tiring to<br />carry and it also responded quicker to the users’ input.<br /><br /><br />Using an iPhone as an input device to control a large display, which<br />GUI and which ordering algorithm would be the most preferable<br />according to the users’ performance and subjective feedback?<br />Three different visualizations were implemented, namely CoverFlow,<br />ZoomGrid and DisplayMap. For the last two visualizations, three<br />different algorithms were used to put objects in order. Therefore, each<br />user tested seven different conditions. A user study was conducted to<br />compare these seven conditions. As the result of the experiment indicated,<br />CoverFlow visualization is significantly slower than the other two GUIs,<br />therefore, it is not appropriate for these sort of tasks.<br /><br /><br />DisplayMap and ZoomGrid were both similarly fast. In fact, no significant<br />difference was indicated. According to the subjective feedback by the<br />users, the ZoomGrid GUI was preferable, because it showed a good<br />overview of the existing objects.<br /><br /><br />What application domain can benefit from the PrIME prototype?<br />To show the application of the PrIME prototype concept in a real life<br />scenario, CrossStorm prototype was implemented. CrossStorm supports<br />users in brainstorming sessions. Users can make, delete, and move<br />the post-its across displays using an iPhone as an input device. This<br />prototype allowed two users to use two iPhones simultaneously, which<br />gave users the possibility of using more iPhones to share their ideas with<br />other members of their group.<br /><br /><br />Lessons learned from the design and implementation of these prototypes<br />showed the impacts of using a mobile input device with an integrated<br />display for cross-display interaction.