Quasi?Experimental Evaluation Method for a Serious Game’s Learning Efficacy

Quasi?Experimental Evaluation Method for a Serious Game’s Learning Efficacy

Wolff, A.J.

Verbraeck, A. (mentor)
Mayer, I. (mentor)
Kortmann, L. (mentor)
Wenzler, I. (mentor)

Technology, Policy and Management

Systems Engineering

SEPAM

2013-04-24

Background: The increase in elaborateness of serious games has resulted in a more and more prominent use in business environments, their use has grown dramatically. Due to this growing interest; Shell has requested, in 2009, the development of the serious game Hazard Recognition, a single player 3D simulation of a Shell drilling site. Motive: The development of Hazard Recognition has reached completion, but due to the absence of experimental evidence that Hazard Recognition contributes to learning, adoption into Shell lacks. In order to collect this evidence, an evaluation study has been proposed. Complication: Mayer et al. (2012) are in the midst of development of an overarching framework for the evaluation of serious games. However, there is no off-the shelve method available for the evaluation of a serious game’s learning efficacy. Therefore, this study’s research question is: “What is an appropriate method, which can operate in the Comprehensive Evaluation Framework (Mayer et al., 2012), for the evaluation of a serious game’s learning efficacy and the contributing factors?” Method: Literature research has provided for the necessary academic foundations for the development of the evaluation method. Successively, a quasi-experimental experiment shaped the context of application and testing of this method. The experiment was the evaluation of Hazard Recognition’s learning efficacy. The result of this experiment showed that Hazard Recognition achieves learning efficacy. Results: An appropriate conceptual method for the evaluation of a serious game’s learning efficacy, within the context of Mayer’s Comprehensive Evaluation Framework, has been laid down. The learning efficacy of a serious game is defined as the ability of the learning intervention to achieve its learning objectives. This can be mapped and measured by evaluating the movement of students through Burch’s Competence Model (1970s). This requires to map the position of the student on the two axes (conscious and competence) by the measurement of self-perceived competences (conscious) and objective performance (competence). For these measurements there has been formulated an experiment setup with various requirements and constructs for the measurement methods have been developed. The case study, which consisted of the evaluation of Hazard Recognition’s learning efficacy, showed that both the developed conceptual model, as the developed measurement methods are valid.

serious game
evluation
learning efficacy
quasi-experimental

http://resolver.tudelft.nl/uuid:73fa6e93-10ff-49c1-a848-eb1169d56f45

2013-04-24

Student theses

master thesis

(c) 2013 Wolff, A.J.