Workplace E-learning, does it Work? A New Approach to Evaluating E-learning Courses with Implications for Design.

Abstract

Do e-learning courses work, and to what extent is the quality of e-courses being examined? From our understanding of current literature, most methods for evaluating e-courses are based on evaluation models that are not intended for e-learning at all. These models mainly evaluate how well a course worked by measuring user satisfaction, changes in behaviour and returns on investment. However, our findings indicate that organizations rarely evaluate properly, if at all. So then, how can we be sure that e-courses work? We consider this problem to be “wicked,” meaning that finding the answer is frustrating and maybe impossible. To suggest an answer, we created a prototype of a new evaluation model. The model means to evaluate e-courses based on how well they facilitate its users’ achievement of learning goals. We named it the LGA (Learning Goal Achievement) Facilitation model. The prototype was created using Design Based Research (DBR) methods. This involved developing a total of four iterations of our model, informed by feedback from members of its target group. We began by creating a draft based on theory, which was then refined into the first iteration of the model and sent to our participants. These participants provided feedback we then used to improve the model by designing new iterations. We ended up with a prototype that was refined three times, and is ready for pilot testing. We found that our participants largely agreed with our approach, and expressed that it could improve practices within their fields. We also found that design guidelines fit into our model, as they could be based on the same principles as evaluation. Making our model as simple and short as possible was considered to make it attractive to use. To know if the model works in practice it would need to be tested, and should be further improved based on test results. We would argue that our work is an example of how students can contribute to practices outside academia, as well as within, and how organizations could benefit from collaboration with students. This model prototype is an original contribution, and the responses we have received indicate a desire for such novelty within the field.