Oral Proficiency Results by User Level
Statistical analysis based on standardized oral proficiency assessments
| Year | Language App | Improved Proficiency One Level Up or more |
Improved Proficiency Two Levels Up or more | Sample Size |
Statistical Report |
|
|---|---|---|---|---|---|---|
| Percent | 95% CI | Percent | ||||
| BEGINNER USERS | ||||||
| ACTFL OPIc Level Improvement | ||||||
| 2019 | 83* | 63-94 | 21 | 24 | ||
| 2019 | Pimsleur | 73 | 61-82 | 10 | 63 | |
| 2018 | italki | 72 | 61-80 | 22 | 81 | |
| 2017 | Hello English | 73 | 60-83 | 16 | 55 | |
| 2016 | Busuu | 75 | 63-85 | 31 | 61 | |
| 2009 | Rosetta Stone | 64** | 56-72 | 19 | 135 | |
| TrueNorth (TNT) Incremental Improvement | ||||||
| 2023 | 89 | 81-94 | N/A | 92 | ||
| 2019 | Mango Languages | 85 | 71-93 | N/A | 46 | |
| INTERMEDIATE USERS | ||||||
| 2021 | 71 | 62-78 | N/A | 109 | ||
* Subsample from users following the Pimsleur Method and completing 30 lessons.
** Not directly comparable because the study time was self-reported.
Explanatory Notes
Methodology and data interpretation guidelines
Data Rounding
The numbers in the table are rounded for clarity and presentation purposes.
Oral Proficiency Assessment
Oral Proficiency Interview (OPI)-based assessment using ACTFL (American Council on the Teaching of Foreign Languages) guidelines. Assessments were conducted by certified evaluators using standardized rubrics.
For computer-based testing, see the Oral Proficiency Interview–Computer (OPIc).
Additionally, assessments like Emmersion.ai provide automated speaking evaluations aligned with ACTFL standards.
italki assessments include one-on-one conversation practice with native speakers, providing unique real-world speaking experience.
Score Gain Measurement
Score Gain: Improvement in oral proficiency score on a 5-point scale (0-5) based on ACTFL proficiency levels, measured over a 12-week study period.
Speaking Practice Hours
Speaking Practice Hours Per Week: Average weekly time spent on active speaking exercises, conversation practice, and oral drills within each app.
Confidence Intervals
95% CI: 95% confidence interval around the mean score gain, indicating the range within which the true population mean likely falls.
Range Definition
Range: Minimum and maximum weekly speaking practice hours observed across all participants in each study group.
