Eyes on the Narrative : Exploring the Impact of Visual Realism and Audio Presentation on Gaze Behavior in AR Storytelling

Weidner, Florian and Hartbrich, Jakob and Arboleda, Stephanie Arévalo and Kunert, Christian and Gerhardt, Christoph and Schneiderwind, Christian and Surdu, Tatiana and Diao, Chenyao and Broll, Wolfgang and Werner, Stephan and Raake, Alexander (2024) Eyes on the Narrative : Exploring the Impact of Visual Realism and Audio Presentation on Gaze Behavior in AR Storytelling. In: ETRA '24: Proceedings of the 2024 Symposium on Eye Tracking Research and Applications :. ACM, New York, pp. 1-7. ISBN 9798400706073

Text (Weidner et al. - 2024 - Eyes on the Narrative Exploring the Impact of Vis)
Weidner_et_al._-_2024_-_Eyes_on_the_Narrative_Exploring_the_Impact_of_Vis.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (1MB)

Abstract

Augmented Reality (AR) and Virtual Reality (VR) are essential tools for researchers and practitioners, serving purposes from training to entertainment: many of these applications rely on agents. This study explores the impact of agent characteristics on user reactions, focusing on gaze as a primary visual attention indicator in AR and VR. While existing research has investigated the agent’s gaze and its influence on the user, it is unclear how the agent’s auralization and visualization influence gaze behaviour. We investigate this by studying the impact of rendering style and type of audio on gaze behaviour during a narrative AR experience. Participants listened to a story with the agent visualized as a cartoon-style or realistic virtual human and auralized with spatial or non-spatial audio. The results revealed that the agent’s rendering style significantly influenced gaze behaviour, with cartoon-style agents capturing more visual attention. Audio variations did not yield significant differences. Together, our findings inform the design of AR user interfaces with agents, suggesting that low-realism visualizations are more captivating and, thus, more suitable for experiences where the user is supposed to look at the storyteller.