Hands-on, Hands-off : Gaze-Assisted Bimanual 3D Interaction

Lystbæk, Mathias and Mikkelsen, Thorbjørn and Krisztandl, Roland and Gonzalez, Eric J. and Gonzalez-Franco, Mar and Gellersen, Hans and Pfeuffer, Ken (2024) Hands-on, Hands-off : Gaze-Assisted Bimanual 3D Interaction. In: UIST'24: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology :. Association for Computing Machinery (ACM), New York, pp. 1-12. ISBN 9798400706288

Text (UIST__24___Mathias____Gaze_Assisted_Bimanual_Interaction_in_3D (4))
UIST_24_Mathias_Gaze_Assisted_Bimanual_Interaction_in_3D_4_.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (4MB)

Abstract

Extended Reality (XR) systems with hand-tracking support direct manipulation of objects with both hands. A common interaction in this context is for the non-dominant hand (NDH) to orient an object for input by the dominant hand (DH). We explore bimanual interaction with gaze through three new modes of interaction where the input of the NDH, DH, or both hands is indirect based on Gaze+Pinch. These modes enable a new dynamic interplay between our hands, allowing flexible alternation between and pairing of complementary operations. Through applications, we demonstrate several use cases in the context of 3D modelling, where users exploit occlusion-free, low-effort, and fluid two-handed manipulation. To gain a deeper understanding of each mode, we present a user study on an asymmetric rotate-translate task. Most participants preferred indirect input with both hands for lower physical effort, without a penalty on user performance. Otherwise, they preferred modes where the NDH oriented the object directly, supporting preshaping of the hand, which is more challenging with indirect gestures. The insights gained are of relevance for the design of XR interfaces that aim to leverage eye and hand input in tandem.