Speech-Augmented Cone-of-Vision for Exploratory Data Analysis

Bovo, Riccardo and Giunchi, Daniele and Sidenmark, Ludwig and Newn, Joshua and Gellersen, Hans and Costanza, Enrico and Heinis, Thomas (2023) Speech-Augmented Cone-of-Vision for Exploratory Data Analysis. In: 2023 ACM CHI Conference on Human Factors in Computing Systems, 2023-04-23 - 2023-04-28, Congress Center Hamburg (CCH).

[thumbnail of chi23b-sub5452-cam-i16]
Text (chi23b-sub5452-cam-i16)
chi23b_sub5452_cam_i16.pdf - Published Version

Download (7MB)


Mutual awareness of visual attention is crucial for successful collaboration. Previous research has explored various ways to represent visual attention, such as field-of-view visualizations and cursor visualizations based on eye-tracking, but these methods have limitations. Verbal communication is often utilized as a complementary strategy to overcome such disadvantages. This paper proposes a novel method that combines verbal communication with the Cone of Vision to improve gaze inference and mutual awareness in VR. We conducted a within-group study with pairs of participants who performed a collaborative analysis of data visualizations in VR. We found that our proposed method provides a better approximation of eye gaze than the approximation provided by head direction. Furthermore, we release the first collaborative head, eyes, and verbal behaviour dataset. The results of this study provide a foundation for investigating the potential of verbal communication as a tool for enhancing visual cues for joint attention.

Item Type:
Contribution to Conference (Paper)
Journal or Publication Title:
2023 ACM CHI Conference on Human Factors in Computing Systems
Uncontrolled Keywords:
Research Output Funding/yes_externally_funded
?? yes - externally fundedno ??
ID Code:
Deposited By:
Deposited On:
07 Mar 2023 10:10
Last Modified:
23 Apr 2024 23:40