Over the past decade, the interest of EdTech companies in digital personalised learning has grown worldwide. This resulted in the development of many digital educational games. That all looks very nice, but what is the impact of these “game-based learning environments” on the learning process of the students who use them? i-Learn researchers took a closer look at this among first-year students, and what did they find? Adaptivity in a digital educational game resulted in faster progress compared to a non-adaptive version of the game. In this blog post, you can read more about the experiment and the implications of the results.
What is an adaptive learning environment?
A learning environment is adaptive when the environment continuously adapts itself to the user. For example, an adaptive learning environment can adjust the difficulty of the next exercise based on the performance on previous exercises. You can read more about this in this blog post.
Why study the impact of adaptivity?
Although it is often assumed that adaptivity and personalisation in digital learning environments will lead to learning gains, there is no scientific consensus to support this claim. On the one hand, only a limited number of studies investigated the impact of adaptivity on leaning gains, and on the other hand, the found results did not always point in one direction.
Researchers within i-Learn saw several possible explanations for the differences in the scientific results. They noted, for example, that there were major differences between the educational games studied, such as which skills the games teach and how adaptivity works exactly within the games.
They also noted that existing research often reduced the impact of adaptivity to additional learning gains: does adaptivity cause pupils to learn more? Yet, the researchers suspected that adaptivity does not primarily have an impact on learning gains, but rather on learning efficiency (how efficiently do students learn, how quickly do they improve?). Therefore, they designed a study to put this hypothesis to the test.
The research: The Number Sense Game
The i-Learn scientists used the Number Sense Game (NSG) for their research. This digital educational game trains numerical skills in young children. It consists of two subgames in which the pupils work with numbers. You can read more details about this game here.
81 pupils of the first grade participated in the study. The pupils practised with the Number Sense Game during six game sessions over a period of three weeks. What the students did not know was that there were two versions of the game: an adaptive version, in which the difficulty of the exercises was adjusted according to the estimated skill level based on the performance on previous exercises (i.e., a personalized approach), and a non-adaptive version in which students were systematically assigned more difficult exercises without considering the student’s previous performance (i.e., a one-size-fits-all approach).
Using a statistical modelling technique (a so-called longitudinal item response model), the children’s progress during the sessions was modelled and assessed. The strength of this progress was in turn compared between the adaptive and non-adaptive version of the game.
The adaptivity ensured that everyone received tailor-made exercises.
What is innovative about this study is that the researchers analysed the log data (i.e. the data generated during game play). This differs from traditional approaches, which only rely on the results of tests taken before and after the game is finished (i.e., a pre-post test design). The use of in-game log data allowed the researchers to assess in game progress during game-play.
The study showed that regardless of the version of the Number Sense Game, the children made progress. However, progress was greater in the adaptive version of the game.
In addition, the researchers found that there were no pupils for whom the game was too difficult or too easy in the adaptive version, in contrast to the non-adaptive version. In other words, the adaptivity ensured that everyone received tailor-made exercises. The i-Learn researchers are convinced that it is important in learning to take into account the individual differences between students in order to stimulate the learning process of all students. This study shows that adaptive learning environments can certainly help to achieve this goal.
Added value for the teacher: more personalised support for the pupil
The results of the study already show the added value of adaptivity for the pupils. But is there also an added value for teachers? The researchers believe so. Adaptive learning environments often keep track of the learner’s skill level, and continuously adjust it based on new actions by the learner. By providing teachers with feedback on a student’s current skill level (e.g. via a dashboard), teachers can quickly assess a student’s level and progress and intervene if necessary.
The teacher can offer new challenges to highly skilled learners and more specialised (personalised) learning activities to less skilled learners. When faster learners are registered, teachers can challenge them with new learning activities, or motivate them to further practice the acquired skills. In this way, they ensure that these students practice at their own level.
Thanks to this continuous feedback and the automatic adjustments based on the student’s previous performance, the teacher can focus more on his or her task as a personal coach, regardless of the student’s level, and thus optimise each student’s learning process.