Validating a group oral task in a university entry test : Interactional competence as a target construct in an academic context

Shahizan, Asbahan and Brunfaut, Tineke (2023) Validating a group oral task in a university entry test : Interactional competence as a target construct in an academic context. PhD thesis, Lancaster University.

Text (2023shahizanphd)
2023shahizanphd.pdf - Published Version
Download (22MB)

Abstract

Increasing seminar-style teaching and assessment diversification in higher education mean that group assessments have become part of degree assessments in many university contexts. In principle, therefore, group oral tasks seem a meaningful task type for university entry tests. However, limited research is available on the validity of such tasks, particularly for local university entrance tests such as the Malaysian University English Test (MUET), of which the scores are used to demonstrate meeting Malaysian university entry requirements. Therefore, this study investigated the interactional features elicited during a group oral task for a university entry test (MUET) and compared them to the interactional features of group oral assessments in the target domain (degree-level study), to shed light on the context validity of MUET’s group oral task. To gain insights into the construct tested by the MUET group oral task versus by degree-level group assessments, video recordings were made of four MUET simulation tests and of two groups of first-year and two groups of final-year students completing assessed group academic discussions in an English language and an IT course, respectively. After transcribing the recordings, Applied Conversation Analysis was conducted to identify the interactional features during group oral performances in the three settings (MUET, English course, IT course). The analyses of the talk revealed differences between the turn and topic management features of the group oral performances in the three contexts with a focus on MUET versus English and IT, and to a lesser extent between groups within the same context. More specifically, differences were found in general features of turn-taking (e.g., organisation of talk in terms of specific strategies used during the initiation, maintaining, and ending of the group interaction), turn length, number of turns, and turn allocations. Topic management features such as opening, topic initiation, extension and closing also differed. In addition, an evaluation of the MUET materials by a group of ten expert language testers suggested that the MUET group interaction task did not lend itself to eliciting all the intended features listed in the MUET test specifications and rating scale. All in all, these findings indicate important shortcomings to the validity of the MUET group oral task. The significance of the study lies in the insights gained into the context validity of the MUET group interaction task and their implications for this high-stakes test. The study also offers insights into the nature of turn and topic management microfeatures in academic group interactions more generally.