Flood, Michelle and Lackovic, Natasa (2018) Shaping up for success : a qualitative case study on the impact of using interactive radar graphs on workplace-based assessment practices for pharmacy interns in Ireland. PhD thesis, Lancaster University.
2018floodphd.pdf - Published Version
Available under License Creative Commons Attribution-NonCommercial-NoDerivs.
Download (8MB)
Abstract
As competency-based health professions education is implemented more widely, the use of workplace-based assessment (WBA) has increased. WBA involves assessment of trainees in the workplace based on observation of performance against structured competency frameworks or using specific WBA tools. In Ireland, pharmacy interns are assessed against a 178-item behavioural checklist by a tutor as part of formative and summative assessments during training. This leads to the generation of large numbers of ratings which may prove challenging to interpret. This thesis aimed to explore how a novel information visualisation tool (Visualisation Tool) designed to support this process and explore how its introduction could impact on WBA practice. An activity theory-based methodology approach was used to first consider current WBA practice (using document analysis and focus groups) before exploring the potential of the Visualisation Tool to influence practice (using a double-stimulation user testing method). The findings indicated that current WBA practice is unexpectedly complex and many challenges were identified. Participants used technology to enter and record ratings, to review ratings, and as a point of reference during review meetings. Using the visualisation addressed problems relating to reductionism, allowed participants to more readily interpret the data, and allowed time in the review meeting to be used more efficiently so that the intern and tutor could prioritise discussion of specific areas of concern. The activity theory-based study design facilitated an in-depth analysis of the role of technology in practice. This study highlighted that technology is one of several, interrelated tools used in WBA and that while technology-based innovations may address some specific issues, a broader, system-level approach is required to address all issues identified as arising in WBA . These issues should be considered in the context of the overall WBA practice rather than in isolation, and researchers should avoid overestimating participants’ estimation of the role of technology.