The relevance of the availability of visual speech cues during adaptation to noise-vocoded speech

Trotter, Anthony S. and Banks, Briony and Adank, Patti (2021) The relevance of the availability of visual speech cues during adaptation to noise-vocoded speech. Journal of Speech, Language, and Hearing Research, 64 (7). pp. 2513-2528. ISSN 1092-4388

[img]
Text (Trotter_submission_R1_marked_Final_accepted)
Trotter_submission_R1_marked_Final_accepted.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.

Download (893kB)

Abstract

Purpose This study first aimed to establish whether viewing specific parts of the speaker's face (eyes or mouth), compared to viewing the whole face, affected adaptation to distorted noise-vocoded sentences. Second, this study also aimed to replicate results on processing of distorted speech from lab-based experiments in an online setup. Method We monitored recognition accuracy online while participants were listening to noise-vocoded sentences. We first established if participants were able to perceive and adapt to audiovisual four-band noise-vocoded sentences when the entire moving face was visible (AV Full). Four further groups were then tested: a group in which participants viewed the moving lower part of the speaker's face (AV Mouth), a group in which participants only see the moving upper part of the face (AV Eyes), a group in which participants could not see the moving lower or upper face (AV Blocked), and a group in which participants saw an image of a still face (AV Still). Results Participants repeated around 40% of the key words correctly and adapted during the experiment, but only when the moving mouth was visible. In contrast, performance was at floor level, and no adaptation took place, in conditions when the moving mouth was occluded. Conclusions The results show the importance of being able to observe relevant visual speech information from the speaker's mouth region, but not the eyes/upper face region, when listening and adapting to distorted sentences online. Second, the results also demonstrated that it is feasible to run speech perception and adaptation studies online, but that not all findings reported for lab studies replicate.

Item Type:
Journal Article
Journal or Publication Title:
Journal of Speech, Language, and Hearing Research
Uncontrolled Keywords:
/dk/atira/pure/subjectarea/asjc/3600/3616
Subjects:
ID Code:
153408
Deposited By:
Deposited On:
01 Apr 2021 11:23
Refereed?:
Yes
Published?:
Published
Last Modified:
14 Jan 2023 02:15