The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1754-2731.htm
Serious games: leverage for Serious games
knowledge management
Caroline Bayart, Sandra Bertezene and David Vallat
University of Lyon 1, Lyon, France, and
235
Jacques Martin
Universite´ du Sud Toulon-Var, La Garde, France
Abstract
Purpose – The purpose of this paper is to investigate if the use of “serious games” with students can
improve their knowledge acquisition and their academic performance.
Design/methodology/approach – The research is an exploratory investigation resorting to the use
of a serious game to evaluate the evolution of the students’ competencies in project management,
through questionnaires processed using a structural “learning model.”
Findings – This research shows indeed that the use of “serious games” improves the knowledge
acquisition and management competencies of the students with the evidencing of significant factors
contributing to this improvement.
Practical implications – The findings of this research show that serious games can be an effective
tool to be used in teaching students particularly as traditional methods are less and less accepted by
today’s students.
Originality/value – Although the use of games is not something new in education, it is still limited
in teaching practices in higher education. This experiment can help lecturers and trainers to resort
to them in their pedagogy and to conceive them according to variables that can enhance their
effectiveness.
Keywords Knowledge management, Knowledge acquisition, Serious games,
Student competency improvement
Paper type Research paper
1. Introduction
The knowledge worker, whose main activity is to think (Davenport, 2005), is a key
actor of the economy of knowledge (Drucker, 1959; Foray, 2009), itself at the heart of
economic growth (Arrow, 1962; Romer, 1994). This is all the more true today as we
witness the development of Creative Common licenses (guaranteeing at the same time
copyright and free circulation; Lessig, 2004) or that of massive open online courses,
MOOC (Friedman, 2013). Knowledge feeds innovation and labor productivity (Powell
and Snellman, 2004; OCDE, 2012a, b), hence the interest of firms and organizations to
invest massively in knowledge (OCDE, 1996, 1999, 2000; Wilson and Briscoe, 2004).
Knowledge and its corollary education are the pillars of competitiveness in the
framework of international competition (World Economic forum, 2012). The role of
education and more particularly of universities consists in helping students in their
apprenticeship so that they become knowledge workers. What teaching methods can
be used to achieve this goal? In order to give some answers to this question, we have
chosen to evaluate the nature of the impacts of a serious game on the acquisition of
knowledge by students. If serious games have attracted attention in previous research,
essentially qualitative (Malone, 1981; Eber, 2003), this research does not allow to The TQM Journal
Vol. 26 No. 3, 2014
identify and understand the mechanisms according to which they influence the process pp. 235-252
r Emerald Group Publishing Limited
of acquisition of knowledge (Egenfeldt-Nielsen, 2007; Wilson et al., 2009). This paper 1754-2731
presents a stage in the research to test the different techniques by professional publics. DOI 10.1108/TQM-12-2013-0143
TQM To carry out our demonstration, we will first analyze the works of Nonaka (1994) in
26,3 the field of knowledge management to understand the implementation of knowledge
thanks to the use of serious games. This approach seems to be particularly relevant
in the context studied as knowledge management is a resource for the firm, but also
for the present or future employee since it is a means of assuring his/her employability
and negotiating power in the firm (Pesqueux, 2010). We will then explain the field
236 research methodology, in our case a survey composed of two questionnaires
administered to nearly 200 students before and after their participation in a serious
game related to project management. The second part presents and analyzes the main
results; the demonstration of the progress in knowledge on the one hand, and the
factors of acquisition of academic knowledge on the other.
2. Theoretical and methodological framework of the research: developing
knowledge thanks to games
The purpose of this first part is to explain the place of serious games in knowledge
management and to explain the game used and the questionnaires in order to know the
role of the serious game in the acquisition of knowledge for the future knowledge workers.
2.1 Serious games at the service of knowledge management
Knowledge management (Nonaka, 1991) consists in making knowledge emerge,
circulate and being seized in order to favor innovation, creativity and to improve the
performance and competitiveness of the organization (Tisseyre, 1999). According to
Nonaka (1994): “Any organization that deals with a changing environment ought not
only to process information efficiently, but also create information and knowledge.”
One the major stakes of knowledge management is the transformation of tacit
knowledge (uses, practices, not necessarily conscious) into explicit knowledge
(formalized and transmissible). This model takes into account not only the
transmission of the different forms of knowledge but also the conversion mechanism
between tacit knowledge and explicit knowledge and conversely.
With this objective in mind, Nonaka (1994) devised the SECI model (socialization,
externalization, combination, internalization) which contains four processes. The first
one, socialization, is about an informal sharing of experience (e.g. between master
and apprentice). The second one, externalization, is about the formalization of tacit
knowledge (through some model of formalization, e.g. a quality approach). The third one,
combination, is about the construction of explicit knowledge from tacit knowledge. The
fourth one, internalization, is the transformation of explicit knowledge into tacit
knowledge through appropriation and akin to learning by doing. Understanding these
different ways of knowledge acquisition in the firm permits to prepare students (the
future knowledge workers) to the stakes of knowledge management, and even to make of
them their own managers of knowledge to develop their action capacity or positive
liberties (Sen, 1999). Information and communication technologies make a new relation
to knowledge possible, where the fundamental question is no longer that of the
acquisition of knowledge but that of its use (Serres, 2012; Oblinger and Oblinger, 2006).
Firms now expect from knowledge managers that they contribute to developing an agile
organization (Sarrazin and Sikes, 2013; Schwaber and Beedle, 2001) which operates in an
uncertain, complex, ambiguous and volatile environment (a VUCA world: volatility,
uncertainty, complexity, ambiguity, Johansen, 2007).
Identifying, ranking, contextualizing, connecting, de-compartmentalizing knowledge
are major competencies for future knowledge workers (Siemens, 2005, 2006; Levy, 2010).
They can thus find intelligibility in the complexity of the world.
Games permit to simulate complex situations. Without going back to the Chinese, Serious games
war games (kriegspiel) have been practiced since the nineteenth century (Perla, 2011).
These games aiming at being trained for decision making in complex situations and
evaluating the impact of the decisions made in order to devise a strategy, have
gradually been applied to the world of business (Gilad, 2009).
Games are at the basis of apprenticeship (ludus in Latin, at the same time the game
and the training place – school) and make them part and parcel of our culture 237
(Huizinga, 1988). We are not considering here an ontological approach of games
(Caillois, 1958; Suits, 2005; Duflo, 1997; Sutton-Smith, 2001), but only see that they have
been used for long in pedagogy with young children (Piaget, 1945) and that they are
more and more used in universities (Sabin, 2007; Young et al., 2012). The use of games
with an educational perspective was formalized as soon as 1970 under the name of
serious games, whose definition was given by Abt (1970): “Reduced to its formal
essence, a game is an activity among two or more independent decision-makers
seeking to achieve their objectives in some limiting context. A more conventional
definition would say that a game is a context with rules among adversaries trying to
reach objectives. We are concerned with serious games in the sense that these games
have an explicit and carefully thought-out educational purpose and are not intended
to be played primarily for amusement.”
The phrase serious games covers a very wide spectrum (Alvarez and Rampnoux,
2007) from advergames (for advertising purposes) to newsgames (for information)
through edutainment (educational games). A serious game is a complex and dynamic
system, composed of objects and actions evolving during the game depending on
the rules and the actions of the players (Thomas et al., 2011). There are two kinds of
games (Sanchez et al., 2011). Either the student plays and acquires knowledge
unrelated to the rules of the game, or the acquisition of knowledge is the very aim of
the game. Serious games permit a contextualized apprenticeship (Shaffer et al., 2005),
favouring autonomy, initiative and complex and critical decisions, especially when
involving numbers (Dickey, 2005).
From the elements provided by the literature, we propose a model aiming at
deepening the study of the link between the interactivity made possible through the
use of serious games and the perceived performance of the students’ apprenticeship.
We try to show that this link is not direct and that the variables (such as the
attractiveness of the pedagogy and the level of interest in the student) interact.
The model is presented in Figure 1.
Perceived quality
of pedagogy
Perceived
Interactivity with
performance of
teachers
learning
Figure 1.
Interest of The research
students model tested
TQM On the basis of this model, we started from the following central hypothesis: serious
26,3 games in Abt’s sense are an effective leverage tool to transform explicit knowledge into
tacit knowledge (learning by doing), leading to a work of appropriation (internalization)
(Nonaka, 1994). Upon this hypothesis, we have different sub-hypotheses:
H1. The interactivity with the teacher proposed during the serious games increases
238 the attractiveness of the pedagogy through games.
H2. The interactivity with the teacher proposed during the serious games increases
the interest of students for the subject taught.
H3. The interactivity with the teacher proposed during the serious games has a
positive impact on the perceived performance of the apprenticeship.
H4. The interest of the students during the game has a positive impact on the
perceived performance of the apprenticeship.
H5. The interest of the students during the game increases the perceived quality of
the pedagogy.
H6. The perceived quality of the pedagogy has a positive impact on the perceived
performance of the apprenticeship.
The following part explains the methodology used to study the progress of knowledge
of the students toward a profile of knowledge worker through a process of
internalization thanks to the game.
2.2 Methodology of collection and analysis of the data
In this part we will present the content and objective of the serious game used and the
methodology used.
2.2.1 A criminal inquiry to learn about project management. Our serious game is not
a free and exploratory game; it is a role game with rules and a finality (Caillois, 1958)
proposed to students in their second year of a two-year degree in “marketing
techniques” at the Institute of Technology of the University of Lyon in France.
“Impossible inquiries” have been chosen by the lecturer both for their entertaining
character and the opportunity they offer to learn about the concepts of project
management. The group of 196 students was divided into 24 teams composed of eight
or nine students. Each team is a detective agency and each agency has got to solve
a case related to one of the two inquiries proposed, to see how knowledge progresses
among the students. The first inquiry is about a hunting accident y or murder
and the objective is to find out who the murderer is and what the motivation is.
The second inquiry is about a girl who has disappeared and the objective is to find out
what happened.
Each inquiry lasts for two days. At the end of each half-day the students have
to report to two lecturers the state of the inquiry. Depending on how they have
progressed, they are given a number of clues. Even if the students do not get the same
clues at the same time, in the end they all have the same clues. These briefing and
de-briefing sessions give the lecturers the opportunity to give the students information
about a number of subjects: how to deal with a lot of data in a short time, how to lead a
project and organize it in an effective way, whether to re-orient or abandon a particular Serious games
project depending on the evolution of the environment. The solution to the riddle is
given to all the students at the same time. Finding the solution is not the main aim of
the game, the work method and the tools used are what is evaluated according to
pre-defined criteria. For each criterion the students provide a written work handed over
to the lecturers at the end of each half-day.
2.2.2 An on-line questionnaire to limit non-responses. Web surveys permit to get in 239
touch with the students rapidly. They are not costly and do not take too much time as
the answers are input directly by the students. The collection and processing of the
information is fast and easy. And this medium also permits a great interactivity and
an automatic control of the validity of the answers. As the students can answer the
questions when and wherever they like, it is also less constraining.
2.2.3 The administration of the questionnaires in two stages. The students have
attended a course on project management before the game so that they are supposed to
apply what they have learnt. That is why we have evaluated their knowledge in this
field before their participation in the game. In total, 27 closed questions were asked
where they could only answer “true or false.” In all, 131 students out of 196 answered
the questionnaire, but only 114 questionnaires were complete and could be exploited; a
final response rate of 58.2 percent.
After the game, the students were asked to answer a second questionnaire in two
parts: the same 27 questions as in the first questionnaire and 21 questions destined to
understand better the perception of the game by the students and how this tool could
have an impact on their apprenticeship. Some general questions were also asked to
identify the sex, the origin (school) and how familiar they were with games. Finally the
response rate was lower than for the first questionnaire; only 99 students tried to
answer the questionnaires (with 14 incomplete responses), the final response being
50.5 percent. More than two-thirds of the respondents to the second questionnaire
(76.2 percent) had also answered the first one.
Females are a little more present in the sample (77 percent) but the difference
is not significant ( p-value ¼ 14.2 percent; see Appendix 1). The origin of the
students is not a discriminating variable either, the distribution of the respondents
is similar to that of the whole group of students ( p-value ¼ 10.8 percent; see
Appendix 2). There is a little significant relation between the grade obtained at
the high-school diploma (baccalaure´at) and the respondents ( p-value ¼ 6.7 percent;
see Appendix 3).
Concerning the academic level of the respondents, no difference appears between
the average obtained by the respondents and the whole group (12.12 vs 12.93,
p-value ¼ 38.7 percent; see Appendix 5). However, the average mark obtained for the
third semester by the respondents is significantly higher than that for the whole group
(12.754 vs 12.33, p-value ¼ 0.23 percent; see Appendix 6). Therefore we can only infer
that the students who had a better mark for the third semester were more willing to
take part in the study.
2.2.4 The “operationability” of the variables. The interaction with the teacher and the
interest of the students during the game are considered as formative variables and
measured according to the scales proposed by Paswan and Young (2002):
Q1: The teacher encourages the students to express their opinions.
Q2: The teacher is receptive to new ideas and different points of view.
Q3: The students have the opportunity to ask questions.
Q4: The teacher stimulates group discussion.
TQM Q1: You were interested in learning the rules of the game.
Q2: You were interested in learning the concepts of project management.
26,3 Q3: You were generally attentive during the game.
Q4: You think that the game was an intellectual stimulus.
Q5: You have become more competent in the field studied (project management).
The perceived quality of the pedagogy and the perceived quality of the apprenticeship
240 are formative variables measured with the indicators proposed by Young et al. (2003).
Globally, this pedagogical game was:
Q1: ineffective/effective;
Q2: useless/useful;
Q3: unsatisfactory/satisfactory;
Q4: bad/good.
Q1: I acquired knowledge in project management.
Q2: I developed competencies in project management.
Q3: I understand project management better.
Q4: I am able to apply the concepts of project management.
Q5: I would like to learn more about project management.
Each question is marked on a Likert scale from 1 (totally disagree) to 7 (totally agree).
Structural equations and their analysis will be used to evaluate this model as we need to
test complex causal models with several latent variables. We will use the PLS method,
based on the analysis of variances, which permits to modelize the data with a series of
multiple regressions translating the existence and the strength of the relations between
the variables. This method is justified by the fact that the research is exploratory, that the
sample size is relatively small and that our model is partial (Tennenhaus, 1998). It is
indeed only part of a more global model where other explanatory variables can be
considered. According to Chin (1998), an empirical rule imposes to have a number of
observations superior to ten times the number of structural relations and ten times the
number of indicators of the most complex formative variable, which is the case in this
study (86 respondents, seven structural relations and five indicators maximum for the
formative variables).
3. Presentation and analysis of the results of the research
The study with the questionnaires provides two types of interesting results. On the one
hand there is a real progress in knowledge whereas the students do not always have the
feeling they acquire competencies. On the other hand we observe that the interactivity and
the pleasure caused by the game could be at the origin of the academic progress observed.
3.1 The progress of tacit knowledge: a step toward internalization
In order to understand the impact of the game on the students’ learning process,
we compare the marks obtained for the test by the students before and after the game.
The results first show that the level of the students in project management is quite
satisfactory (19-20 good answers to the 27 questions). Then we observe that the
knowledge of the students has improved during the game. Based on the sample of
students who answered both questionnaires, the average mark has significantly
increased ( p-value ¼ 1 percent; Appendix 7). This is in keeping with the conclusions of
experiments carried out in the literature. Various studies show indeed that games allow
a progress in knowledge (Shaffer et al., 2005; Dickey, 2005; Simon, 2005) and also of
procedural knowledge, that is competencies that the student can hardly describe but
that he can master (Sanchez, 2011).
However, by segmenting the sample of respondents into two groups, we observe Serious games
that the knowledge of the students whose academic mark was lower (inferior to the
group’s average mark) have not progressed during the game (mark from 20.6 to 20.2,
not significant). But the students whose mark was higher than the group’s average
have progressed (19.56-20.84, p-value ¼ 00.4 percent; Appendix 8). So it seems that the
game is mainly profitable for the “good” students.
The marks clearly show the progress in knowledge. The students feel that they 241
have been intellectually stimulated during the game and the majority of them recognize
having been very attentive during it. Yet few think that they have gained more
knowledge and competencies with the game (36 percent). These results are confirmed
by the analysis of the perceived benefits of the game. A third of the students feel that
they understand project management better and would like to learn more but only
19 percent think they have acquired competencies and less than 17 percent think they
have acquired knowledge in the subject at the end of the game. This paradox could be
explained by three factors (Bierly et al., 2000): the teachers did not explain sufficiently
the objective of the game so students cannot evaluate their results properly; the
teachers operated too small a transfer of knowledge during the game, so the students
feel they did not learn much; the limited organizational capacity and the lack of method
of the students limit the feeling of progress. The gap between the objective progress
and the perceived progress could be reduced by acting on the different parameters of
the game with a better integration of the pedagogical aspects into the game (Malone,
1981; Malone and Lepper, 1987; Habgood et al., 2005; Egenfeldt-Nielsen, 2006), the
improvement of the sequences of the game, notably the “de-briefing” sessions. Fanning
and Gaba (2007) showed that this stage is essential for the effectiveness of serious
games as it permits to move from knowledge applicable to the game, to knowledge
applicable to different contexts. De-briefing is then a major tool in the externalization
process.
3.2 Interactivity and enjoyment: two factors of knowledge acquisition
In the survey after the game we tried to know how the students perceived the game.
We asked them 21 questions classified under five themes. Each question is marked on
a Likert scale from 1 (absolutely agree) to 5 (absolutely disagree). The items were
borrowed from tested and validated work in cognitive psychology.
Among those questions, some aimed at measuring the interaction between
students on the one hand, and between students and teachers on the other. Such
information is useful to appreciate the positive impact of games in relationships
with others (Simon, 2005). This interactivity between students and with teachers
allows the implementation of an “active” pedagogy where the student contributes
to learning by participating in the creation of knowledge (Blasco-Arcas et al., 2013).
It has been demonstrated that students learn better when they take part in the
learning process (Prince, 2004). Knowledge and experience are shared so that
students develop interdependence fostering a critical outlook and encouraging them
to participate, explain and justify their point of view. Therefore they are more
committed, and enjoy playing the game. This commitment has got many
aspects, which can be behavioral, emotional and cognitive (Fredericks et al.,
2004). In any case this commitment is, at least partly, determined by the relations
an individual establishes with the environment. Thus the change generated
by the methods of active pedagogy modifies the perception of the students and
impacts their level of commitment. Commitment is an important variable in the
TQM learning process (Ahlfeldta et al., 2005; Furlong and Christenson, 2008). This
26,3 situation increases the probability that the whole group learns project management
(Soller, 2001).
The interactivity was perceived by three-fourth of the respondents. Our model
explains more than 29 percent of the students’ interest for the game and it is precisely
interactivity with the teacher that explains this result (the “perceived difficulties”
242 variable is not significant, Table AI). Interactivity also plays an important role in the
perceived quality and performance of the learning process. The model explains 51.2
percent of the perceived quality of the pedagogy where the students’ interest has the
strongest impact followed by the interactivity with the teacher (Table AII). The model
also explains 65 percent of the perceived performance of the learning process with
three variables contributing significantly: the students’ interest and the perceived
quality of the pedagogy followed by once again the interactivity with the teacher
(Table AIII).
Another point deserves being underlined: the students enjoyed playing the game.
In total, 91 percent found it entertaining and 69 percent found it satisfactory. Playing
a game causes pleasure among the players (Shaftel et al., 2005) as it offers the
possibility to interact, to collaborate, to feel committed and to face challenges while
controlling the situation (Sanchez et al., 2011). It is thanks to these virtues that games
are relevant to fight against school failure and pathologies (Wilson et al., 2006). In our
case, the pleasure to play stimulated motivation and acted as leverage to acquire
knowledge.
3.3 The benefits of the research: reliability and validity of the variables
The methodology usually used to test the model comprises three stages: ensure that
the supposed links between the variables exist, test the validity of the measurement
model, then of the structural model in order to verify the hypotheses formulated.
The significance of the coefficients is evaluated following a bootstrap, a method
consisting in replicating the estimate of the model on a big number of samples
randomly constructed from the data collected (here, 205 samples of 86 individuals).
Table I sums up the essential characteristics of the model.
The correlations between the latent variables of our model supposed to be linked
exist and are significant.
In the case of the formative variables, it is not necessary to test the internal
coherence of the measurement scales with Cronbach’s a coefficient as the indicators are
supposed not to co-vary (Nunnally and Bernstein, 1994). Nevertheless we need to
analyze the cross-loadings matrix which corresponds to the results of an exploratory
analysis with principal components in order to check that the most important factorial
weights of each indicator are really linked to the corresponding variable. This is
Correlations
Interactivity Interest Perceived Performance
with of quality of of learning
Mean SD teachers students pedagogy process
Table I. Interactivity with teachers 2.09 0.755 1
The essential Interest of students 2.49 0.977 0.371 1
characteristics Perceived quality of pedagogy 2.35 0.954 0.419 0.664 1
of the model Performance of learning process 3.15 0.845 0.286 0.549 0.53 1
globally the case in our study (except for the variables ECH2, ATT4 and INTER3, but Serious games
the gaps are not very big) as shown in Table II.
We also check that the indicators significantly contribute to the formative construct.
For that, we observe the critical ratio, which corresponds to the value of the t-test
which must be more than two to be statistically significant (Table III).
After the bootstrap procedure, we see that three indicators have non-significant
contributions in relation to their latent variable (ECH2, ATT4, INTER3). This could 243
Interactivity Attractiveness Interest of Performance of
with teachers of pedagogy students learning process
ECH1 0.537 0.264 0.245 0.172
ECH2 0.112 0.127 0.006 0.013
ECH3 0.673 0.317 0.275 0.266
ECH4 0.755 0.302 0.343 0.313
ATT1 0.363 0.963 0.662 0.735
ATT2 0.398 0.808 0.605 0.545
ATT3 0.173 0.333 0.293 0.191
ATT4 0.15 0.186 0.216 0.05
INTER1 0.119 0.336 0.341 0.151
INTER2 0.236 0.382 0.529 0.39
INTER3 0.312 0.175 0.284 0.107
INTER4 0.237 0.439 0.529 0.322
INTER5 0.452 0.668 0.975 0.759 Table II.
PERF1 0.285 0.655 0.647 0.851 Links between the
PERF2 0.346 0.534 0.68 0.82 most important
PERF3 0.415 0.604 0.592 0.83 factorial weights and
PERF4 0.26 0.666 0.642 0.847 the corresponding
PERF5 0.3 0.382 0.456 0.585 latent variables
Manifest External weight Critical
Latent variable variables (Bootstrap) SD ratio (CR)
Interactivity with teachers ECH1 0.479 0.226 2.38
ECH2 0.079 0.247 0.452
ECH3 0.606 0.142 4.756
ECH4 0.692 0.162 4.669
Attractiveness of pedagogy ATT1 0.946 0.03 32.287
ATT2 0.791 0.081 9.963
ATT3 0.322 0.13 2.562
ATT4 0.171 0.187 0.998
Interest of students INTER1 0.294 0.147 2.317
INTER2 0.509 0.12 4.404
INTER3 0.247 0.155 1.834
INTER4 0.499 0.137 3.863
INTER5 0.956 0.031 31.61
Performance of learning process PERF1 0.838 0.061 13.871
PERF2 0.808 0.064 12.777
PERF3 0.804 0.059 14.004
PERF4 0.832 0.068 12.459 Table III.
PERF5 0.572 0.088 6.62 Critical ratios
TQM be due to the fact that the phrasing of the items may not be quite relevant or that the
26,3 students did not quite understand them.
The PLS method gives the Goodness of Fit (GoF) as indicator of adjustment of
the model. The nearer 1 the latter is, the more the model is confirmed by the data.
Finally we must check that the values before and after the bootstrap are not too
different to conclude if the model is stable (Table IV).
244 The GoF is 0.432, very close to its bootstrap estimate. The relative GoF and those
based on the internal and external models are very high and tend to translate a good
quality of adjustment of the model to the data.
Once the measurement model has been studied, the structural model must be analyzed
to validate the hypotheses of the study. This validation depends on the importance and the
significance of the structural relations obtained. The determination coefficients (R2) account
for the explained variance of the latent variables. Here too, they must be more than two,
which is shown in the results (Table AI, AII, AIII) and which is summed up in Figure 2.
Apart from the progress in knowledge, there is a link between the interactivity
proposed by the game and the perceived performance of the learning process, but this
relation is not very strong. The quality of the pedagogy and the interest of the students
for the game also contribute to explain the perceived performance. The relations are
complex and other variables can contribute to explain the perceived quality of the
learning process, such as the pleasure to play.
4. Conclusion
Students are future “knowledge workers” that can be trained through “serious games”
used as leverage for transforming explicit knowledge into tacit knowledge and develop
learning by doing. The model we have used shows the link between interactivity and
perceived performance of the learning process. The results show progress of the
students in the subject studied. This progress is due, at least partly, to the interaction
among students and with teachers, as well as the enjoyment the students derived from
GoF GoF (Bootstrap) SD Critical ratio (CR)
Absolute 0.432 0.445 0.032 13.674
Relative 0.782 0.751 0.049 16.073
Table IV. External model 0.869 0.855 0.041 21.164
GoF Internal model 0.9 0.878 0.032 28.356
Perceived quality of
pedagogy
R 2=51.2%
Perceived
Interactivity with performance of
teachers learning
R 2=65%
Figure 2.
The results of Interest of students
the research R 2=29.5%
the game. The quality of the pedagogy and the interest of the students for the game also Serious games
contribute to explain the perceived performance. There should be, however, other
variables that ought to be investigated to explain this progress in knowledge acquisition.
It would also be of interest to compare the results obtained here with other teaching
methods used for the same subject.
References
245
Abt, C. (1970), Serious Games, The Viking Press, New York, NY.
Ahlfeldta, S., Mehta, S. and Timothy, S.T. (2005), “Measurement and analysis of student
engagement in university classes where varying levels of PBL methods of instruction are
in use”, Higher Education Research & Development, Vol. 24 No. 1, pp. 5-20.
Alvarez, J. and Rampnoux, O. (2007), “Serious game: just a question of posture?”, Proceedings of
Artificial & Ambient Intelligence, AISB’07, Newcastle, April, pp. 420-423.
Arrow, K.J. (1962), “The economic implications of learning by doing”, Review of Economic Studies,
Vol. 29 No. 3, pp. 155-173.
Bierly, P.E., Kessler, E.H. and Christensen, E.W. (2000), “Organizational learning,
knowledge and wisdom”, Journal of Organizational Change Management, Vol. 13
No. 6, pp. 595-618.
Blasco-Arcas, L., Buil, I., Hernandez-Ortega, B. and et Sese´, F.J. (2013), “Using clickers in class.
The role of interactivity, active collaborative learning and engagement in learning
performance”, Computers and Education, Vol. 62, March, pp. 102-110.
Caillois, R. (1958), Les jeux et les hommes, Gallimard, Paris.
Chin, W.W. (1998), “The partial least squares approach to structural equation modeling”, in
Marcoulides G.A. (Ed.), Modern Methods for Business Research, Lawrence Erlbaum
Associates, Mahwah, NJ, pp. 295-336.
Davenport, T.H. (2005), Thinking for a Living: How to Get Better Performance and Results from
Knowledge Workers, Harvard Business Press Books, Cambridge, MA, 240pp.
Dickey, M. (2005), “Engaging by design: how engagement strategies in popular computer and
video game can inform instructional design”, Educational Technology Research and
Development, Vol. 53 No. 2, pp. 67-83.
Drucker, P. (1959), Landmarks of Tomorrow, Harper & Brothers, New York, NY.
Duflo, C. (1997), Jouer et philosopher, PUF, Paris.
Eber, N. (2003), “Jeux pe´dagogiques – Vers un nouvel enseignement de la science e´conomique”,
Revue d’e´conomie politique, Vol. 113 No. 4, pp. 485-521.
Egenfeldt-Nielsen, S. (2006), “Overview of research on the educational use of video game”, Digital
kompetanse, Vol. 1 No. 3, pp. 184-213.
Egenfeldt-Nielsen, S. (2007), Educational Potential of Computer Games, Continuum,
New York, NY.
Fanning, R. and Gaba, D. (2007), “The role of debriefing in simulation-based learning”, Simul
Healthcare, Vol. 2 No. 1, pp. 1-11.
Foray, D. (2009), L’e´conomie de la connaissance, La De´couverte, Paris.
Fredericks, J.A., Blumenfeld, P.C. and Paris, A.H. (2004), “School engagement: potential of
the concept, state of the evidence”, Review of Educational Research, Vol. 74 No. 1,
pp. 59-109.
Friedman, T. (2013), “The professors’ Big Stage”, New York Times, March 5, p. A23, available at:
http://ick.li/TqicgQ (accessed May 25, 2013).
TQM Furlong, M. and Christenson, S. (2008), “Engaging students at school and with learning:
a relevant construct for all students”, Psychology in the Schools, Vol. 45 No. 5,
26,3 pp. 365-368.
Gilad, B. (2009), Business War Games, Career Press, Franklin Lakes, NJ.
Habgood, M.P.J., Ainsworth, S.E. and Benford, S. (2005), “Endogenous fantasy and learning in
digital game”, Simulation and Gaming, Vol. 36 No. 4, pp. 483-498.
246 Huizinga, J. (1988), Homo Ludens, Gallimard, Paris.
Johansen, B. (2007), Get There Early: Sensing the Future to Compete in the Present,
Berrett-Koehler Publishers Inc, San Francisco, CA.
Lessig, L. (2004), Free Culture, The Penguin Press, New York, NY.
Levy, F. (2010), “How technology changes demands for human skills”, OECD Education Working
Papers, No. 45, OECD Publishing, Paris.
Malone, T.W. (1981), “Toward a theory of intrinsically motivating instruction”, Cognitive Science,
Vol. 5 No. 4, pp. 333-369.
Malone, T.W. and Lepper, M.R. (1987), “Making learning fun: a taxonomy of intrinsic motivations
for learning”, in Snow, R.E. and Farr, M.J. (Eds), Lawrence Erlbaum Associates, Hillsdale,
NJ, pp. 223-253.
Nonaka, I. (1991), “The knowledge-creating company”, Harvard Business Review, Vol. 69 No. 6,
pp. 96-104.
Nonaka, I. (1994), “A dynamic theory of organizational knowledge creation”, Organization
Science, Vol. 5 No. 1, pp. 14-37.
Nunnally, J.C. and Bernstein, I.H. (1994), Psychometric Theory, 3rd ed., McGraw-Hill,
New York, NY.
Oblinger, D. and Oblinger, J. (Eds) (2006), “Is it age or IT: first steps toward understanding the net
ge´ne´ration”, Educating the Net Generation, available at: www.educause. edu/ir/library/
pdf/pub7101b.pdf (accessed May 25, 2013).
OCDE (1996), L’e´conomie fonde´e sur le savoir, OCDE, Paris.
OCDE (1999), L’Economie fonde´e sur le savoir : des faits et des chiffres, OCDE, Paris.
OCDE (2000), Socie´te´ du savoir et gestion des connaissances, OECD Publishing, Paris.
OCDE (2012a), Connected Minds: Technology and Today’s Learners, Educational Research and
Innovation, OECD Publishing, Paris.
OCDE (2012b), Etudes ´economiques de l’OCDE, 2012/8, No. 8, OCDE, Paris.
Paswan, K.A. and Young, J.A. (2002), “Student evaluation of instructor: a nomological
investigation using structural equation modeling”, J Mark Educ, Vol. 24 No. 3,
pp. 193-202.
Perla, P. (2011), The Art of Wargaming: A Guide For Professionals and Hobbyists, John Curry, US
Naval Institute, Annapolis, MD.
Pesqueux, Y. (2010), “Du knowledge management a` la socie´te´ de la connaissance”, HAL:
hal-00509676, version 1.
Piaget, J. (1945), La formation du symbole chez l’enfant. Imitation, jeu et re#ves, Delachaux &
Niestle´, Neuchaˆtel.
Powell, W.W. and Snellman, K. (2004), “The knowledge economy”, Annual Review of Sociology,
Vol. 30, pp. 199-220.
Prince, M. (2004), “Does active learning work? A review of the research”, Journal of Engineering
Education, Vol. 93 No. 3, pp. 223-231.
Romer, P.M. (1994), “The origins of endogenous growth”, The Journal of Economic Perspectives, Serious games
Vol. 8 No. 1, pp. 3-22.
Sabin, P. (2007), Lost Battles: Reconstructing the Great Clashes of the Ancient World, Hambledon
Continuum, London.
Sanchez, E. (2011), “Usage d’un jeu se´rieux dans l’enseignement secondaire : mode´lisation
comportementale et e´piste´mique de l’apprenant”, Revue d’intelligence artificielle, Vol. 25
No. 2, pp. 203-222. 247
Sanchez, E., Ney, M. and Labat, J.M. (2011), “Jeux se´rieux et pe´dagogie universitaire : de la
conception a` l’e´valuation des apprentissages”, Revue internationale des technologies en
pe´dagogie universitaire, Vol. 8 Nos 1/2, pp. 48-57.
Sarrazin, H. and Sikes, J. (2013), “Competing in a digital world: four lessons from the software
industry”, McKinsey & Cie, available at: www.mckinsey.com/insights/business_
technology/competing_in_a_digital_world_four_lessons_from_the_software_industry
(accessed May 25, 2013).
Schwaber, K. and Beedle, M. (2001), Agile Software Development With Scrum, Prentice Hall,
Upper Saddle River, NJ.
Sen, A. (1999), L’e´conomie est une science morale, La De´couverte, Paris.
Serres, M. (2012), Petite Poucette, E´ditions Le Pommier, Paris.
Shaffer, D.W., Squire, K.R., Halverson, R. and Gee, J.P. (2005), “Video games and the future of
learning”, Phi Delta Kappan, Vol. 87 No. 2, pp. 104-111.
Shaftel, J., Pass, L. and Schnabel, S. (2005), “Math games for adolescents”, Teaching Exceptional
Children, Vol. 37 No. 3, pp. 25-30.
Siemens, G. (2005), “Connectivism, a learning theory for the digital age”, available at: www.
ingedewaard.net/papers/connectivism/2005_siemens_ALearningTheoryForTheDigital
Age.pdf (accessed May 25, 2013).
Siemens, G. (2006), “Knowing knowledge”, available at: www.elearnspace.org/KnowingKnowledge_
LowRes.pdf (accessed May 25, 2013).
Simon, L. (2005), “Du gamesman aux gamers : le jeu comme dynamique e´mergente des organisations
contemporaines?”, Gestion, Vol. 30 No. 4, pp. 34-42.
Soller, A.L. (2001), “Supporting social interaction in an intelligent collaborative learning
system”, International Journal of Artificial Intelligence in Education, Vol. 12 No. 1,
pp. 40-62.
Suits, B. (2005), The Grasshopper: Games, Life and Utopia, Broadview Press, London.
Sutton-Smith, B. (2001), The Ambiguity of Play, Harvard University Press, Cambridge, MA.
Tennenhaus, M. (1998), La re´gression PLS, the´orie et pratique, Technip, Paris.
Thomas, P., Yessad, A. and Labat, J.M. (2011), Petri Nets and Ontology: Tools for the Learning
Player Assessment in Serious Games, ICAL, Athens, GA.
Tisseyre, R.-C. (1999), Knowledge management, the´orie et pratique de la gestion des
connaissances, Rene´-Charles Tisseyre, Herme`s.
Wilson, A.J., Dehaene, S., Pinel, P., Revkin, S.K., Cohen, L. and Cohen, D. (2006), “Principles
underlying the design of “The Number Race”, an adaptive computer game for remediation
of dyscalculia”, Behavioral and Brain Functions, Vol. 2 No. 19, pp. 2-16.
Wilson, K.A., Bedwell, W.L., Lazzara, E.H., Salas, E., Burke, C.S., Estock, J.L. and Conkey, C.
(2009), “Relationships between game attributes and learning outcomes, review and
research”, Simulation & Gaming, Vol. 40 No. 2, pp. 217-266.
Wilson, R.A. and Briscoe, G. (2004), “The impact of human capital on economic growth: a
review”, in Descy, P. and Tessaring, M. (Eds), Impact of Education and Training, Third
TQM report on vocational training research in Europe: background report Office for Official
Publications of the European Communities, Luxembourg, pp. 9-71.
26,3
World Economic Forum (2012), Global Competitiveness Report, World Economic Forum, Geneva.
Young, M.F., Scota, S., Cutter, A.B., Jalette, G., Mullin, G., Lai, B., Simeoni, Z., Tran, M. and
Yukhymenko, M. (2012), “Our princess is in another castle: a review of trends in serious
gaming for education”, Review of Educational Research, Vol. 82 No. 1, pp. 61-89.
248 Young, M.R., Klemz, B.R. and Murphy, J.W. (2003), “Enhancing learning outcomes: the effects
of instructional technology, learning styles, instructional methods, and student behavior”,
J Mark Educ, Vol. 25 No. 2, pp. 130-142.
Further reading
Gruson, B., Forest, D. and Loquet, M. (2012), Jeux de savoir, Presses universitaires de Rennes,
Rennes.
Siemens, G. and Tittenberger, P. (2009), Handbook of Emerging Technologies for Learning,
available at: http://techcommittee.wikis.msad52.org/file/view/HETL.pdf (accessed May
25, 2013).
Thie´tart, R.A. (Ed.) (1999), Me´thodes de recherche en management, Dunod, Paris, pp. 226-238.
Wacheux, F. (1996), Me´thodes qualitatives et recherche en gestion, Economica, Paris, pp. 201-209.
Appendix 1. Sex of the respondents
Population 1: 132
Sample size 1: 196
Population 2: 49
Sample size 2: 64
Supposed difference (D): 0
Variance: p1q1/n1 þ p2q2/n2
Significance level (%): 5
z-Test for two proportions/bi-lateral test:
95 percent confidence interval around the difference of proportions:
] 0,215; 0,031 [Difference 0.092
z (observed value) 1.471
z (critical value) 1.960
p-value (bi-lateral) 0.141
a 0.05
Interpretation of the test:
H0 : The difference between the proportions is equal to 0.
Ha : The difference between the proportions is different from 0.
Given that the p-value calculated is superior to the a significant threshold ( ¼ 0.05), the
null hypothesis H0 cannot be rejected. The risk of rejecting the null hypothesis H0 if it is true, is
14.03 percent.
Appendix 2. School origin of the respondents
Independence test (w2):
w2 (observed value) 4.453
w2 (critical value) 5.991
DDL 2
p-value 0.108
a 0.05
Given that the p-value calculated is superior to the a significant threshold ( ¼ 0.05), the Serious games
null hypothesis H0 cannot be rejected. The risk of rejecting the null hypothesis H0 if it is true, is
10.79 percent.
Appendix 3. The grade at the baccalaure´at of the respondents
Independence test (w2):
w2 (observed value) 5.414
w2 (critical value) 5.991 249
DDL 2
p-value 0.067
a 0.05
Given that the p-value calculated is superior to the a significant threshold ( ¼ 0.05), the
null hypothesis H0 cannot be rejected. The risk of rejecting the null hypothesis H0 if it is true, is
6.67 percent.
Appendix 4. Having a high grade at the baccalaure´at
Population 1: 148
Sample size 1: 196
Population 2: 57
Sample size 2: 64
Supposed difference (D): 0
Variance: p1q1/n1 þ p2q2/n2
Significance level (%): 5
z-Test for two proportions/bi-lateral test:
Confidence interval (95 percent) around the difference of proportions:
] 0,233; 0.038 [
Difference 0.136
z (observed value) 2.729
z (critical value) 1.960
p-value (bi-lateral) 0.006
a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we must reject
the null hypothesis H0, and retain the alternative hypothesis Ha. The risk of rejecting the null
hypothesis H0 if it is true is inferior to 0.63 percent.
Appendix 5. The mark in project management
Descriptive statistics:
Observations Observations with missing data Observations without missing data
Missing variables Minimum Maximum Mean SD
Total Nico 196 0 196 0,000 16,050 11,916 1,931
Comparison 64 0 64 8,990 15,490 12,122 1,546
z-Test for two independent samples/bi-lateral test:
95 percent confidence interval around the difference of means:
] 0.671; 0.260 [
Difference 0.206
z (observed value) 0.866
|z| (critical value) 1.960
p-value (bi-lateral) 0.387
a 0.05
Given that the p-value calculated is superior to the a significant threshold ( ¼ 0.05), the
null hypothesis H0 cannot be rejected. The risk of rejecting the null hypothesis H0 if it is true,
is 38.66 percent.
TQM Appendix 6. The average mark in the third semester
z-Test for two independent samples/uni-lateral test ( left):
26,3 95 percent confidence interval around the difference of means:
] -Inf; 0.168 [
Difference 0.402
z (observed value) 2.832
250 z (critical value) 1.645
p-value (uni-lateral) 0.002
a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we must reject the
null hypothesis H0, and retain the alternative hypothesis Ha. The risk of rejecting the null
hypothesis H0 if it is true is inferior to 0.23 percent.
Appendix 7. Evaluation
z-Test for one sample/uni-lateral test (right):
95 percent confidence interval around the mean:
] 0.216; þ Inf [
Difference 0.734
z (observed value) 2.330
z (critical value) 1.645
p-value (uni-lateral) 0.010
a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we must reject the
null hypothesis H0, and retain the alternative hypothesis Ha. The risk of rejecting the null
hypothesis H0 if it is true is inferior to 0.99 percent.
Appendix 8. Evaluation according to the segments
t-Test for two paired samples/uni-lateral test ( left):
95 percent confidence interval around the difference of the means:
] -Inf; 0.661 [
Difference 1.250
t (observed value) 3.568
t (critical value) 1.681
DDL 43
p-value (uni-lateral) 0.000
a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we must reject the
null hypothesis H0, and retain the alternative hypothesis Ha. The risk of rejecting the null
hypothesis H0 if it is true is inferior to 0.04 percent.
Appendix 9. Marks of girls vs boys (S3)
t-Test for two independent samples/bi-lateral test:
95 percent confidence interval around the difference of the means:
] 0,035; 1.157 [
Difference 0.596
t (observed value) 2.122
|t| (critical value) 1.999
DDL 62
p-value (bi-lateral) 0.038
a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we must reject the Serious games
null hypothesis H0, and retain the alternative hypothesis Ha. The risk of rejecting the null
hypothesis H0 if it is true is inferior to 3.78 percent.
Appendix 10. Marks for the game for girls
Observations Observations with missing data Observations without missing data
Missing variables Minimum Maximum Mean SD
Mark before 49 0 49 12,000 24,000 19,878 2,705 251
Mark after 49 0 49 13,000 25,000 20,857 2,566
t-Test for two paired samples/bi-lateral test:
95 percent confidence interval around the difference of the means:
] 1,675; 0.285 [
Difference 0.980
t (observed value) 2.834
|t| (critical value) 2.011
DDL 48
p-value (bi-lateral) 0.007
a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we must reject the
null hypothesis H0, and retain the alternative hypothesis Ha. The risk of rejecting the null
hypothesis H0 if it is true is inferior to 0.67 percent.
Appendix 11. Marks for the game for boys
Observations Observations with missing data Observations without missing data
Missing variables Minimum Maximum Mean SD
Mark before 15 0 15 14,000 24,000 20,000 2,268
Mark after 15 0 15 16,000 23,000 19,933 2,492
t-Test for two paired samples/bi-lateral test:
95 percent confidence interval around the difference of the means:
] 1,464; 1,597 [
Difference 0,067
t (observed value) 0.093
|t| (critical value) 2.145
DDL 14
p-value (bi-lateral) 0.927
a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we cannot
reject the null hypothesis H0. The risk of rejecting the null hypothesis H0 if it is true is
92.69 percent.
Appendix 12. Use of video games
Independence test (w2):
w2 (observed value) 21.651
w2 (critical value) 7.815
DDL 3
p-value o0.0001
a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we must reject
the null hypothesis H0, and retain the alternative hypothesis Ha. The risk of rejecting the null
hypothesis H0 if it is true is inferior to 0.01 percent.
TQM Appendix 13. Use of stage games
Independence test (w2):
26,3 w2 (observed value) 1.844
w2 (critical value) 7.815
DDL 3
p-value 0.605
252 a 0.05
Given that the p-value calculated is inferior to the a significance level ( ¼ 0.05), we cannot
reject the null hypothesis H0. The risk of rejecting the null hypothesis H0 if it is true is
60.55 percent.
Interest of students R2 R2 (Bootstrap) Critical ratio (CR)
Table AI. 0.223 0.295 2.956
Structural model: Structural coefficients Value Value (Bootstrap) Critical ratio (CR)
interest of Interactivity with teachers 0.428 0.454 5.413
the students Perceived difficulties 0.163 0.155 0.898
Perceived quality of pedagogy R2 R2 (Bootstrap) Critical ratio (CR)
Table AII. 0.479 0.512 6.38
Structural model: Structural coefficients Value Value (Bootstrap) Critical ratio (CR)
perceived quality Interactivity with teachers 0.294 0.313 6.817
of pedagogy Interest of students 0.507 0.502 10.852
Performance of apprenticeship R2 R2 (Bootstrap) Critical ratio (CR)
0.631 0.66 10.749
Table AIII. Structural coefficients Value Value (Bootstrap) Critical ratio (CR)
Structural model: Interactivity with teachers 0.199 0.216 6.757
perceived performance Interest of students 0.373 0.369 13.272
of apprenticeship Perceived quality of pedagogy 0.364 0.361 14.498
Corresponding author
Dr Jacques Martin can be contacted at: jma.martin@wanadoo.fr
To purchase reprints of this article please e-mail: reprints@emeraldinsight.com
Or visit our web site for further details: www.emeraldinsight.com/reprints
