Project summary: The assessment gap: The case of Det Norske Veritas (Maritime) Problem to be addressed: The proposed research project will identify the gap between the existing situation and the desired situation when it comes to how Det Norske Veritas Maritime (DNV M) is using evaluation strategies on competence development for the employees. DNV is an international foundation and is a provider of services for risk management. I will focus on DNV M department which is specialists in technology and business risks consulting for customers in the maritime industry. The employees are basically engineers who are going through continuous competence development to be able to perform services of risk management. However, there is a lack of focus on evaluation of the learning processes and outcomes of the training programs. The project will highlight a part of this knowledge gap by first describe the existing competence development procedures offered at DNV M for the employees. I am focusing on an “assessment of the evaluation procedures” mainly using four evaluation strategies designed by one of the main proponents when it comes to evaluation, Donald Kirkpatrick. I will then present how training courses are evaluated; to identify which stage DNV M is in today according to Kirkpatrick’s four level evaluation strategy:
1) Measuring reaction: it measures the customer’s satisfaction after a training program.2) Measuring learning: “the extent to which participants change attitudes, improve knowledge, and/or increase skills as a result of attending the program” (Kirkpatrick 1998 p.20). 3) Measuring behaviour: the extent to which change in behaviour has occurred because the participants attended the training program.4) Measuring result: The final results can include increased production, improved quality, decreased costs, reduced frequency and/or severity of accidents, increased sales, reduced turnover and higher profits (Kirkpatrick 1998 p.23).
The next step is to describe the desired situation of the training programs and evaluation methods:
1) A desired situation according to ten interviewees/employees at DNV,2) A desired situation according to DNV Learning and DNV Maritime Learning,3) A desired situation according to evaluation literature and organizational learning literature. The evaluation literature will be linked to- and analyzed in view of organizational learning literature.
Methods: I have chosen basically an empirical approach- and theoretical approach to this thesis. A critical theory perspective will guide me. Kirkpatrick’s four evaluation strategies will be highlighted though the thesis as a framework to understand and measure the evaluation situation. I will raise the evaluation discussion through an analytical understanding of evaluation as a tool of organizational learning. The thesis is an empirical contribution where I am exemplifying with DNV as a case study. The empirical data is analyzed through theoretical literature both evaluation literature and organizational learning literature. I will do this through three main research activities: 1) a literature review of evaluation; 2) a comprehensive review and analysis of DNV written documents about competence development strategies and evaluation procedures; 3) interviews of ten key persons who work in- and with evaluation at DNV Maritime.
Significance of the project: DNV`s objectives are safeguarding life, property, and the environment. Failure to manage risks can be dangerous; consistency and control are becoming even more critical. DNV can help to manage this new risk in reality. With continued focus on competence development of the employees, the evaluation procedures to assess the competence development need to be present. Otherwise, if the competence development procedures are not satisfactory and are not being evaluated, this could have serious implications on the outcome of performance and quality of risk management. The training programs should continuously be evaluated and reconsidered as a quality and safety routine. This thesis highlights the discussion about how it is possible to use evaluation as a tool in organizational learning.
Findings: I can see a tendency of level 1 evaluation, and in some training courses- level 2 evaluation. It has indeed been accomplished two projects where the four levels were used, but otherwise there are no formal and continuous procedures of level 2, 3 or 4 evaluation. Even though level 1 evaluation is almost present in all the training courses, the procedures are different, and the focus on evaluation is fluctuating. The visions of DNV as a holder of the best competence in their field and how DNV M document their competence in reality can indicate a gap in two ways:1) What the different levels (HR- and training managers, line managers and employees) at DNV M state and what happens in reality. 2) The existing situation of evaluation at DNV M and how evaluation literature (i.e. Kirkpatrick) and describes evaluation and how organizational learning literature describes a learning organization.
Anticipated benefits: I will use the findings to reflect on procedures for improving the focus on evaluation. These recommendations can be utilized by several business departments at DNV. The findings can also be seen as a contribution to the evaluation literature, with special focus on Kirkpatrick’s evaluation model. The discussion about evaluation as a tool in organizational learning can contribute to broaden the understanding of how to use, perform and reflect around evaluation in organizations.