H2E : Hand, Head, Eye a Multimodal Cascade of Natural Inputs

Patel, Khushman and Phadnis, Vrushank and Gonzalez, Eric J and Gellersen, Hans and Pfeuffer, Ken and Gonzalez-Franco, Mar (2025) H2E : Hand, Head, Eye a Multimodal Cascade of Natural Inputs. In: 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) :. IEEE, pp. 84-89. ISBN 9798331525637

Text (GEMINI_-_2025_fina-without-changes_3 (1))
GEMINI_-_2025_fina-without-changes_3_1_.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (2MB)

Abstract

Eye-based interaction techniques for extended reality, such as gaze and pinch, are simple to use however suffer from input precision issues. We present H2E, an integrated fine and coarse-grained pointing framework that cascades Hand, Head, and Eye inputs. We further introduce the MagicPinch gesture as an example of the framework, which allows for smooth transitioning between fine and coarse-grained pointing. When combined together, after users initiate a pinch gesture, a cursor appears midway during the pinch at the position of the gaze, which can be dragged by head pointing if needed before pinch confirmation. This has the advantage that it can add a precision component without changing the semantics of the technique. In this paper, we describe the design of the H2E framework and implementation of the MagicPinch technique. Furthermore, we present an evaluation of our method in a Fitts-based user study, exploring the speed-accuracy trade-offs against a gaze and pinch interaction baseline.