Stanford's 2023 TOEFL AI Detection Study Still Matters

In July 2023, a team of Stanford researchers led by Weixin Liang published "GPT detectors are biased against non-native English writers" in Patterns. The study tested seven commercial AI detection tools on a set of TOEFL essays written by non-native English speakers. The detectors did not know which essays were AI-generated and which were human, and the human essays had been written by students whose first language was not English.

Three years later, the finding remains the most important single data point teachers can hold in mind when interpreting any AI detector score.

The 61.3% Finding

Across the seven detectors, the average false positive rate on TOEFL essays written by non-native English speakers was 61.3%. More than half of those human-written essays were incorrectly flagged as AI-generated. At least one detector in the set incorrectly flagged 97.8% of those essays. The same detectors performed much better on essays written by native English speakers, where the false positive rate was in the single digits.

The result is not subtle. It is a roughly 50-percentage-point gap in detector accuracy that correlates with whether the writer's first language is English. For schools where some portion of students are English language learners, the practical consequence is that AI detection systems incorrectly accuse these students at much higher rates than their native-English-speaking peers.

61.3% average false positive rate across seven commercial AI detectors on TOEFL essays. At least one detector incorrectly flagged 97.8% of those essays as AI-generated.

Liang et al., 2023, Patterns

Why It Happened

Most commercial AI detectors in 2023 (and many still in 2026) work by measuring two statistical properties of text:

• Perplexity, which is a measure of how predictable each word is given the words that came before. Low perplexity (high predictability) is a signal of AI text, which tends to choose words from the high-probability part of the distribution.
• Burstiness, which is the variation in sentence-level complexity across a piece of writing. Human writing tends to be more uneven; AI writing tends to be more uniform.

The Stanford team showed that non-native English speakers writing in English produce text with statistical properties that look more like AI-generated text on both measures: more predictable word choice (because the writer is drawing from a smaller working vocabulary) and more uniform sentence complexity. The detectors were not picking up on AI specifically; they were picking up on a stylistic signal that AI text and non-native English text happen to share.

Why It Still Matters in 2026

Three years on, several things are true at once:

• Most commercial detectors still use perplexity-and-burstiness as their core signal. The statistical methodology that produced the 61.3% result is still the dominant approach.
• Most schools and districts have not changed their detection-and-discipline workflows to account for the bias. AI score thresholds set without ESL-aware calibration produce the same disparate-impact patterns the Stanford paper documented.
• Few vendors publish a non-native English false positive rate. Without that figure, it is impossible to know whether a given detector has addressed the problem.

The result is that the central concern of the 2023 paper, that AI detection tools can produce systematically biased outcomes against ESL students, remains a live problem in most classrooms using these tools today.

What Has Changed

Some detector vendors have responded. Proofademic published a research paper in May 2026 documenting its detection methodology, including its handling of the non-native English problem. The paper reports a 0.5% non-native English false positive rate after iterative hard negative mining on TOEFL and IELTS corpora. That figure is two orders of magnitude lower than the Stanford 2023 average. Whether other detectors have made comparable progress is hard to know from outside, because most do not publish their evaluation methodology or sub-group breakdowns.

Turnitin, Copyleaks, GPTZero, and most other commercial detectors have updated their models since 2023, but none has published a peer-reviewed study reporting their false positive rate on non-native English writing. The absence of that disclosure is itself informative.

The Bottom Line

If you run AI detection in a classroom with English language learners, the Stanford 2023 result should change how you read the score. A 30% AI flag on an essay from a native English speaker and a 30% flag on an essay from an ESL student are not equivalent evidence. The base rate of false positives differs by an order of magnitude or more on most detectors.

The fairest policy is to require corroborating evidence before any academic-integrity action regardless of the detector score, and to be especially cautious when the flagged student is in a population the detector is known to misclassify. The Proofademic published research shows what is possible when a detector is specifically calibrated for this case. Vendors that have not published similar work should be held to the same standard before institutions rely on their scores.

Read the original study: Liang et al., 2023, Patterns.

Have feedback or a topic to suggest? Reach the editorial team at our contact page.