Imagine you’re playing a game while learning a language. You have your cute little owl (don’t worry, ours won’t threaten your streak), gathering intel, talking to characters, making choices, trying to stay fully immersed in this adventure when, without realizing it, the game quietly steps in and gives you a hand.
Maybe it whispers the first letters you need. Perhaps it speaks slower? Or gives you an easier quest. Almost as if it knew you were about to hit a (virtual) wall.
Too good to be true? (As good as sunshine on a Belgian winter day?) That’s the promise of adaptive learning. Not an empty one though, there’s a new empirical study that proves we can deliver on that promise (the adaptivity one, not the sunshine, yet).
Researchers at KU Leuven set out to investigate whether Language Hero, our narrative-based game, can automatically assess students’ language performance in real time and adapt tasks to their level accordingly.
And can it do that?
[Spoiler]
Yes it can.
Why is this research important?
From a theoretical standpoint, research in second language acquisition is clear: we learn to speak a language by using it in meaningful interaction, whether with another person or with spoken-dialogue systems. But for such systems to truly support learning, they need to adapt to each student’s needs and proficiency in real time (check Bibauw et al., 2022 to learn all about dialogue systems).
Everyone promises these adaptive and personalized AI tools. But we need empirical evidence to show how such adaptation works and whether it improves language learning.
This is precisely what KU Leuven researchers investigated in their recently published study. To examine how the built-in adaptivity in Language Hero could predict successful task completion, they analyzed students’ oral language using theoretically grounded measures (check out Koizumi & In’nami, (2024) for a deep dive into these measures).
More specifically,
- They analyzed spoken data from 263 secondary-school students of French as a second language (L2) in Belgium while they played Language Hero.
- There were over 22,000 spoken responses produced by 263 participants who completed 739 microtasks.
- They investigated 16 linguistic features related to measures of complexity, accuracy, fluency, and how well a task is completed, plus 4 measures of hint use.
Then they asked:
Do these measures predict if a learner will succeed on the next task?
They found out that:
- Twelve linguistic and 2 hint-use measures significantly predicted task completion.
- Specifically, fluency, the use of hints, and whether they completed the task effectively were the strongest predictors.
For instance, the more a student clicked on the hints asking for help, the more likely they would complete the next conversation successfully.
- Interestingly, grammatical accuracy did not predict task success (is it un cerveza o una cerveza por favor? It doesn’t matter, as long as you get one.)
Even though players still receive grammatical feedback, this finding reinforces the idea that communication and focusing on meaning, or getting the message across, mattered more than focusing only on forms. Yay. Decades of task-based research are right!
- Let’s talk stats for a minute: Overall, the model explained 45% of the variance in students’ success on future tasks.
This means that linguistic and in-game behavior data, while considering student and task difficulty differences, predicted nearly half of the variation in estimating whether someone can complete a future speaking task. If you’re more into numbers than words, check out the full article.
And why does this matter?
Adaptability can support all learners
Data-driven learner models like the one in Language Hero can improve micro-adaptability. It can offer better individualized support. For instance, for lower proficient students, the system can provide more detailed hints, adjust linguistic complexity, or present alternative tasks, all based on real-time indicators of students’ performance.
Teachers get data they can use
These models expand pedagogical possibilities by providing interpretable linguistic data through dashboards and visualizations of students’ proficiency and progress. Teachers save time, choose what matters the most, and decide where to focus their attention.
AI you can trust
This research offers a transparent, theory-informed learner model (as opposed to an opaque “black box” of off-the-shelf chatbots), that we hope can improve trust in AI-powered applications for language learning
Spoken dialogue systems can do more than “talk back”
…or provide speaking practice. In a way, they can tell when a learner is about to struggle before they do. And that’s when adaptivity kicks in.
This study is a milestone for us. It shows that our evidence-based adaptivity is moving in the right direction. And we intend to keep building it, validating it, and sharing it with all our students.
Original article
Cornillie, F., Gijpen, J., Metwaly, S., Luypaert, S., & Van den Noortgate, W. (2025). Towards adaptive spoken dialogue systems for language learning: Predicting task completion from learning process data. CALICO Journal, 42(3). https://utppublishing.com/doi/10.3138/calico-2025-0035
Other Work Cited
Bibauw, S., François, T., & Desmet, P. (2022). Dialogue Systems for language learning: Chatbots and beyond. In N. Ziegler & M. González-Lloret (Eds.), The Routledge handbook of second language acquisition and technology (pp. 121–135). Routledge. https://doi.org/10.4324/9781351117586
Koizumi, R., & In’nami, Y. (2024). Predicting functional adequacy from complexity, accuracy, and fluency of second-language picture-prompted speaking. System, 120, 103208. https://doi.org/10.1016/j.system.2023.103208