A learning analytics model for measuring and evaluating the impact of training programmes in organisations

Velandia, John and Oztok, Murat (2023) A learning analytics model for measuring and evaluating the impact of training programmes in organisations. PhD thesis, Lancaster University.

Text (2023velandiaphd)
2023velandiaphd.pdf - Published Version
Available under License Creative Commons Attribution-NonCommercial-NoDerivs.
Download (5MB)

Abstract

Measuring and evaluating the impact of training programmes in organisations has been challenging for decades. To tackle this challenge, plenty of evaluation models have been proposed to assess the grade of satisfaction, learning, application and return on investment that training brings to organisations. Notwithstanding, these models are not designed to present data so that decision-makers can evaluate the effectiveness of training programmes using learning analytics. Hence, this study aims to design a learning analytics model (LAM) that provides the missing analytics piece of the existing evaluation models. In addition, this investigation should answer the research question: How can a learning analytics model provide relevant data to measure and evaluate the impact of training programmes in organisations? In addition, the research strategy proposed by this study is based on theoretical research as a research methodology. The Thematic Analysis (TA) method was adopted to analyse qualitative data from existing literature, research studies and books. The proposed LAM was built on the basics of the Theory Development Process, from which three building blocks were defined: elements, relationships, and assumptions. In this manner, a solid model can be understood and implemented successfully in organisations. The LAM poses five themes that cluster the identified elements and relationships according to their nature: data sources, external and internal factors, measures, metrics and indicators, data preparation and reporting. Compared to the existing analytical models, this model's novelty comprises the elements required to measure and evaluate the impact of training in organisations by considering perspectives of the data generated before, during and after learning processes are delivered. Those perspectives involve elements of different natures, for example, data sources, external factors that impact the organisation, learning processes, learning experience, stakeholders, business goals, financial goals, analytical models, measures, metrics and indicators. In addition, this model establishes the conditions and assumptions for those who desire to implement or replicate the LAM effectively. Further studies may adopt the proposed LAM to measure and evaluate the impact of training by implementing reports or dashboards based on their organisational needs. Furthermore, future studies also may explore other dependencies among the elements to identify and widen the number of relationships proposed by this investigation