AI Text Detectors: What the Independent Research Says

The independent research on AI text detectors says, with unusual consistency, that they are not accurate enough to settle a question of authorship on their own. A 2023 test of fourteen tools found none above 80% accuracy. A 2023 study of seven detectors found they flagged more than half of essays by non-native English writers as machine-written. OpenAI withdrew its own classifier in July 2023 for low accuracy. A 2024 benchmark of six million generations found detectors easily fooled by a paraphrase or a changed sampling setting. And the one 2025 study that reports a detector at 100% never tested it on human writing.

That is the summary. The rest of this post is the evidence, study by study, because the number that matters depends on which question you are asking. An educator deciding whether to confront a student needs the false-positive rate on human text. A content team deciding whether a client's detector score means anything needs to know how the score responds to light editing. A compliance lead needs to know what a vendor has actually published versus claimed. The studies answer different questions, and the differences are the point.

One scope line. Our post on whether human editing can make writing look AI-generated covers the newer research on editing and detector scores. This one is about the broader accuracy evidence from 2023 to 2025, and it does not repeat that material.

Five sources carry this post. Three are academic papers with stated samples and methods, one is a vendor's own launch and withdrawal page, and one is a 2025 journal article that measured detection but not false positives. They span the period from the first ChatGPT-era detectors to the current generation, which matters because Turnitin's model alone has been updated repeatedly since 2023, and the older results describe tool versions that no longer run.

The widest early test is "Testing of Detection Tools for AI-Generated Text" by Weber-Wulff and colleagues, submitted to arXiv on June 21, 2023 and later published in the International Journal for Educational Integrity. Nine researchers prepared 54 test documents across classes: human-written, ChatGPT-generated, edited, machine-translated and obfuscated. Each document went through 12 publicly available tools and two commercial systems, Turnitin and PlagiarismCheck, and the results were scored for accuracy, false positives and false negatives.

The abstract's conclusion is blunt. The authors write that the available detection tools "are neither accurate nor reliable" and that they carry "a main bias towards classifying the output as human-written rather than detecting AI-generated text". In the discussion, all tools "scored below 80% of accuracy and only 5 over 70%". Turnitin did best, with 41 of 54 documents correct under strict scoring, which is 76%, or 79% with partial credit. The human-written class was the easy one: tools averaged 94% on it. Accuracy fell sharply as obfuscation was added, down to 30% and 15% on the most manipulated classes.

Why it matters: this is the study that established the shape every later one confirms. Detectors are good at calling human text human, worse at catching machine text, and much worse once the machine text has been touched. The limits are equally clear. Fifty-four documents is a small set, the text came from spring-2023 ChatGPT, and the tools were tested in April and May 2023 in versions that have since been replaced several times. The study describes the problem, not today's products.

Two months earlier, Liang and colleagues had asked a narrower question with a sharper answer. Their paper, "GPT detectors are biased against non-native English writers", submitted April 6, 2023 and later published in the journal Patterns, ran 91 human-written TOEFL essays and 88 US eighth-grade essays through seven widely used detectors. The eighth-grade essays came from a public dataset; the TOEFL essays from a Chinese educational forum.

The same mechanism cuts the other way. The abstract notes that simple prompting strategies can both reduce the bias and effectively bypass the detectors, because asking a model for more varied language raises perplexity on machine text as readily as fluent writing lowers it on human text. The failure case for anyone using a detector on a mixed population is therefore specific and documented: the writers most likely to be falsely accused are the ones writing in a second language.

Limits, again stated plainly: 91 essays from one source, 2023 detectors, and Turnitin was not among the seven. Detector vendors have updated their models since, and nothing we read re-tested the current models on the same question. The bias was measured once, clearly; whether it persists is not something the published evidence answers.

The clearest single data point comes from the company with the most to gain from a working detector. On January 31, 2023 OpenAI launched an AI text classifier and published its own evaluation on a challenge set of English text: the classifier correctly identified 26% of AI-written text as likely AI-written, while labelling human-written text as AI 9% of the time. The same page warned that it "should not be used as a primary decision-making tool" and was very unreliable on texts under 1,000 characters.

Six months later the page gained one more line. As of July 20, 2023, the classifier "is no longer available due to its low rate of accuracy". That is the entire stated reason; OpenAI published no post-launch evaluation, and we do not infer one. What the episode establishes is a floor for the whole category: a lab with full knowledge of how its own models write, and every incentive to ship a detector, measured a one-in-four catch rate and a roughly one-in-eleven false-accusation rate, and decided that was not good enough to keep online.

The largest test is RAID, by Dugan and colleagues, submitted to arXiv on May 13, 2024: over six million generations across 11 models, 8 domains, 11 adversarial attacks and 4 decoding strategies, run against 8 open-source and 4 closed-source detectors. Its methodological contribution is the one every detector comparison should adopt: accuracy is reported at a fixed 5% false-positive rate, because a detector can look accurate on machine text simply by calling everything machine.

RAID also maintains a public leaderboard. We read it on November 5, 2026, and the visible tab shows aggregate scores near 1.0 for the top entries, including a commercial detector, an academic system and a grammar vendor's tool. We do not quote those as accuracy figures, because the page text does not say which metric or which attack setting the tab reports, entries are largely self-submitted, and the paper's own adversarial results are far lower. A leaderboard number without its setting is not a result.

The limit of RAID is the limit of any benchmark: it compares machine text to a fixed human corpus, while real writers edit, mix and rewrite. Whether eleven automated attacks resemble what a student or a copywriter actually does is contested, and the paper does not claim it.

The most recent peer-reviewed study we read is by Muhammad Abid Malik and Amjad Islam Amjad, published in the Journal of Applied Learning and Teaching on January 12, 2025. Four tools, Turnitin, ZeroGPT, GPTZero and Writer AI, were run on text from ChatGPT, Perplexity and Gemini, then on the same text after three treatments: a Grammarly edit, a Quillbot paraphrase, and 10 to 20% editing by a human expert.

The abstract reports that Turnitin "turned out to be the most accurate and consistent one, with a 100% AI score even with the adversarial techniques", that paraphrasing hurt the other three tools the most, and that Gemini-generated text drew lower AI scores than Perplexity's. The authors also flag an inconsistency that any user of these tools will recognise: in many cases, files generated by the same model and checked by the same tool returned different scores.

Here is why the study cannot be read as a vindication. Every document in it was machine-generated. A tool that scores 100% on AI text under edits would also score 100% by calling everything AI, and the abstract reports no test of human-written text. The false-positive rate, the number that decides whether a student is wrongly accused, is not measured. Put beside Weber-Wulff's 2023 result, where the same vendor's tool was the best of fourteen at 76%, the two studies are not in conflict; they measured different models on different text with different metrics.

Turnitin is the vendor whose false-positive figure is most often repeated, so its own statements deserve exact quotation, labelled as what they are. On March 16, 2023, a Turnitin blog post stated that the company's efforts "have primarily been on ensuring a high accuracy rate accompanied by a less than 1% false positive rate", and in the same post that "given that our false positive rate is not zero", instructors will need to apply professional judgment. A June 14, 2023 post by Annie Chechitelli of Turnitin refined the number.

Three things about those sentences. First, the document-level figure carries a condition: it applies to documents with 20% or more AI writing, which means the rate on a document that is entirely human is not what the headline says. Second, the 4% is the chance that a highlighted sentence is actually human-written, and the post says this is more common in documents that mix human and AI text, which describes most edited work. Third, no sample, method or test set is given on either page, so neither number can be checked.

The model has also changed since. Turnitin's release notes, read on November 5, 2026, carry an entry dated October 14, 2026 stating that the company improved detection in its English model "while minimizing false positives". No new rate accompanies it. The practical consequence is that every independent test above describes a model that no longer runs, and the current model has no independent false-positive measurement we could find. That is the gap the hero's fourth number records.

Four findings survive every study. Independent tests since 2023 have not found a detector accurate enough to settle authorship alone. The one study in this review that tested non-native writers found them misclassified at far higher rates. Simple manipulation lowers detection in every study that tried it. And the vendor most associated with low false positives says its rate is not zero and leaves the decision to a human.

Three questions remain open. Whether Turnitin's current model performs as the vendor states on fully human text has no independent answer. Whether RAID-style automated attacks resemble real editing is argued both ways. And whether the 2023 bias against non-native writers persists after retraining has not been re-measured in anything we read. The router below is how we would act on that state of evidence, depending on who is asking.

The publisher row is the one with a future. Detection is an arms race the published evidence says the detectors are losing; provenance is a record the author controls. Our guide to experience signals under E-E-A-T and our walkthrough of Google's open-source C2PA validator cover the two halves of that, and our content strategy post on automation and authenticity is the editorial policy that sits above both. If you want that policy written into a production workflow, it is part of what our content engine service builds.