When Liang et al. (2023) published its analysis of seven commercial AI detectors applied to TOEFL essays written by non-native English speakers, the finding was blunt: an average false-positive rate of 61.3 percent. Native-speaker essays from a U.S. eighth-grade sample were flagged at roughly 5 percent. Same tools, same prompt classes, dramatically different treatment.
Quick Answer
Most major detectors still do not publish subgroup false-positive rates for non-native English writers. A handful have raised aggregate accuracy claims. One newer academic vendor has published a specific non-native number in a methodology paper. Districts serving ESL populations should treat subgroup disclosure as a procurement requirement, not a nice-to-have.
We walk through where the research left the market in 2023, what has changed on the vendor side, and what a serious procurement standard would look like in 2026.
Where We Were in 2023
The mechanism was linguistic, not conspiratorial. Non-native writers, especially those preparing for standardized tests, tend to use more predictable syntax, more common vocabulary, and less lexical variety. Detectors that scored “human” partly on perplexity and burstiness read those patterns as machine-generated.
The paper landed in the middle of the first ChatGPT panic. Districts had bought detectors on the assumption that they were neutral instruments. The Stanford result argued they were the opposite of neutral for the students most likely to be disadvantaged by a false accusation.
The students misflagged by an unaudited detector do not care about the vendor’s IP strategy.
The Working Educators editors
What Vendors Have Published Since
Three years on, the honest answer is: not much. The major detectors have shipped model updates. Some have raised their aggregate accuracy claims. Very few have published subgroup false-positive rates for non-native English writers.
Turnitin updated its model and now publishes an aggregate false-positive rate on its Turnitin's AI FAQ. It has not, in materials we have reviewed, released a comparable subgroup number for ESL populations. GPTZero has published training-data descriptions and an aggregate rate. Its subgroup disclosure is limited. We compare the two on this axis in Turnitin vs GPTZero.
Copyleaks, Originality.ai, Winston AI, and Sapling publish aggregate numbers on their marketing pages. None have released the peer-reviewable evaluation set that would let a district audit the subgroup claim. That is not proof they are worse than they say. It is proof buyers cannot check.
The Detector That Named the Number
Proofademic published a methodology paper in 2026 that reports a 0.5 percent false-positive rate on non-native English samples, alongside a 0.2 percent rate on general academic writing. The paper documents the evaluation set and the sentence-level scoring approach.
We mention this descriptively rather than as an endorsement. Self-reported numbers are self-reported. What matters for this piece is that a vendor was willing to disclose the number at all. That single act of transparency raises the bar for the rest of the market.
If Proofademic’s number holds up under independent evaluation, it suggests the disparate-impact problem is technically tractable when a vendor deliberately calibrates against academic and multilingual samples. If it does not hold up, the paper still provides the specification an independent researcher would need to test it. Either outcome is better than the current market default of silence.
Why Most Vendors Won’t Publish
There are three plausible reasons vendors do not release subgroup false-positive rates. First, the number is bad and disclosure would hurt sales. Second, the vendor has not measured it because it costs money to build a properly stratified evaluation set. Third, the vendor considers the evaluation set a trade secret.
The first two reasons should worry every district serving ESL, EAL, or newcomer populations. The third reason is defensible in principle and useless in practice: districts cannot audit trade secrets, and the students misflagged by an unaudited detector do not care about the vendor’s IP strategy.
After Vanderbilt disabled Turnitin’s AI detector in 2023, cited above, other institutions followed. The Stanford paper was one of the reasons. The vendors most exposed to that reputational risk are the ones with the most to gain from publishing a defensible subgroup number and the most to lose from stalling.
What Districts Serving ESL Populations Should Require
We would put four requirements in any procurement RFP for an AI detector at a district with a meaningful multilingual population. One: a published false-positive rate on native English academic writing. Two: a published false-positive rate on non-native English writing, with the evaluation sample described. Three: a documented appeals process for students who dispute a flag. Four: a public commitment to update both numbers with each model release.
Vendors that meet all four are, to our knowledge, rare. Vendors that meet the first three are more common than they were in 2023 but still a minority. Vendors that meet none of the four should not be on the shortlist for a district where a single false flag on a newcomer student can undo a year of trust with a family.
For districts that want to go further, our detector evaluation guide lays out the three questions to ask any vendor before signing. And for the pedagogical alternative, see our editorial on why detection-first integrity fails. For measured accuracy by tool, see our comparison of 12 AI detection tools.
Frequently Asked Questions
Has the Stanford finding been replicated?▼
Independent researchers have reproduced the direction of the effect. The magnitude varies by detector, evaluation set, and model generation. The core finding of disparate impact on non-native writers has not been overturned.
Are newer detectors better on this problem?▼
Some are meaningfully better on general academic writing. Whether they are better on non-native writing specifically depends on whether they trained on multilingual academic samples and whether they published the resulting numbers.
What should a district ask a vendor?▼
Published false-positive rates for native and non-native English, a described evaluation sample, a documented appeals process, and a commitment to update the numbers with each model release.
Should ESL teachers use detectors at all?▼
That is a values question, not a technical one. If the district uses one, the teacher should know its documented failure modes and never treat a flag as evidence sufficient for a conduct decision.
Where can I find the original Stanford paper?▼
Liang et al. 2023 was published in Cell Patterns and is linked in the body of this article.
The Bottom Line
Three years after the Stanford finding, the market has moved further on aggregate accuracy claims than on subgroup disclosure. That mismatch is the disparate-impact problem hiding in plain sight.
Districts serving multilingual populations do not need to leave the detector market to protect their students. They do need to raise the bar. Ask for the subgroup number, ask for the evaluation set, and treat vendors that refuse to provide either as vendors that have not earned the contract. For teachers choosing a tool on their own, our guide to the best AI detector for teachers weighs subgroup disclosure in its ranking.
