Playing While Learning?
A UX Research Study on Gamification Features
Role
Lead Researcher
Skills
Heuristic Evaluation, Usability Testing, Think-Aloud Protocol
Timeline
2023
Context
Academic research, Indiana University Bloomington
Overview
Duolingo is the world's most popular language app, built around streaks, XP, and leagues. We ran a two-part study: a heuristic evaluation with two long-term users, and think-aloud usability testing with two first-time learners. Both groups hit the same wall. The app is excellent at quizzing and weak at teaching. Errors get corrected but never explained, and there is no way to review a grammar rule or look up a word you have already met. Four participants total — two experts, two novices. The split was the point: it let me see which problems survive familiarity and which only new users hit.
The Duolingo Dilemma
Popularity
The world's #1 language app, loved for its game-like design.
The Hypothesis
Gamification may be crowding out instruction.
The Core Question
Does the game support the learning, or overshadow it?
Our Two-Pronged Investigation
The Expert Review
Heuristic Evaluation
- • 2 long-term expert users
- • Nielsen's 10 Heuristics + custom principles
- • Fun & Enjoyment heuristic
- • User Loyalty heuristic
The Novice Experience
Usability Testing
- • 2 first-time users learning French
- • Think-Aloud protocol
- • Real-time confusion capture
- • No prior familiarity bias
The Veteran & The Rookie
Veterans
User A
1.5 years of daily use, current active user
User B
Former user, quit after a broken streak destroyed motivation
Rookies
User C
No prior French experience, first time using Duolingo
User D
No prior French experience, first time using Duolingo
Veteran's Findings — Depth Over Dailies
Key Issues
No Big Picture
Users cannot track overall progress or view a list of mastered vocabulary. There is no way to see what you've actually learned versus what you've just completed.
Feedback Without Context
Errors are corrected but never explained. There are no grammar rules, no pronunciation guides, no structural explanation of the language.
The Stress of the Streak
Gamification transitions from motivator to anxiety source. The streak becomes an obligation rather than a reward, shifting focus from learning to maintenance.
“I want to do this to learn languages, but now I'm just doing it to keep the streak going. It wasn't satisfying.”
— User B, Veteran (former user)
Rookie's Findings — A Confusing First Day
Key Issues
Disorienting Start
No tutorial or guided introduction. Users are dropped straight into a lesson. One said they felt "startled." They don't understand the rules, the scoring, or the goal.
Ambiguous Icons
Key UI elements lack labels. The headphone icon, for example, gave no indication of its function. Users had to guess at every interaction.
No Easy Undos
In sentence-building tasks, there is no way to correct or undo a placed word without starting over. This creates friction in an exercise designed around trial and error.
Core Insight
The app excels at quizzing but fails at teaching the fundamental structure of a language.
Impact on Rookies
Lessons felt like a 'waste of time,' relying on guessing rather than understanding. Without grammar foundations, correct answers felt accidental.
Impact on Veterans
Forced to leave the app and use Google to understand their own mistakes. The app couldn't explain why an answer was wrong.
Both groups asked for the same two things: grammar rules and a personal dictionary. The error states below are the proof.
Critical Product Question
How might Duolingo evolve from a 'game that teaches' into a 'learning tool that is fun'?
Building the Bridge
Guided Onboarding
A tutorial or guided tour before the first test — introducing what XP, hearts, leagues, and streaks mean, and how lessons work.
Actionable Feedback
Contextual grammar explanations at the point of error, not just the correct answer. Show users why they were wrong, not just that they were.
A Knowledge Hub
A Grammar Notes and Personal Dictionary section where users can review rules and vocabulary they've encountered — bridging the gap between quizzing and teaching.
Outcome
Before
No grammar reference
Errors corrected, never explained
No vocabulary list
No way to review what you learned
No onboarding
Dropped straight into a lesson
After
Contextual grammar
Explanation at the point of error
Personal Dictionary
Plus Grammar Notes
Guided tour
Before the first test
Reflection
This project taught me that engagement isn't the same as effectiveness. Duolingo's gamification is very well engineered for retention — but retention and learning are different outcomes, and optimizing for one can sometimes undermine the other. As a researcher, it taught me to always ask "effective at what?" when evaluating a product feature. It also demonstrated the value of combining expert review with user testing. The heuristic evaluation identified potential issues quickly and cheaply; the usability test revealed which of those issues actually mattered to real users and surfaced problems that no heuristic framework would predict.
Thanks for reading!