- Why Detecting AI-Generated Content Is So Hard
- 1. People Cannot Reliably Tell
- 2. Detector Tools Make Costly Mistakes
- 3. Detection Is Easy to Evade
- 4. Watermarks Help, but Only When Built In
- 5. Provenance Flips the Question
- What to Do Instead of Trusting a Score
- Key Terms for Detecting AI-Generated Content
- Quick Questions, Answered
- Final Thoughts: Verify the Source, Not the Style
Detecting AI-generated content sounds like a technical problem with a technical answer: paste the text into a detector, read the score, done. The research says otherwise. People are poor at spotting machine-written text, automated detectors make serious mistakes, and those mistakes fall hardest on writers who have done nothing wrong.
That does not mean nothing works. It means the question is changing, from “was this made by AI?” to “can this content show where it came from?” This guide, part of our Computational Communication section, sets out five facts the evidence supports and what to do with them.
Key takeaways
- Humans perform close to chance when asked to tell AI-written text from human-written text.
- In one well-known test, AI detectors wrongly flagged 61.3% of essays by non-native English writers on average.
- Light rewording can defeat most detectors, so determined misuse is the hardest kind to catch.
- Watermarks and Content Credentials are more promising because they record origin at the moment of creation.
Why Detecting AI-Generated Content Is So Hard
Language models are trained to produce text that is statistically similar to human writing. Each new generation gets closer. A detector is trying to find a gap that the model’s developers are working to close.
Most detectors rely on signals such as how predictable the wording is. Text that is very smooth and predictable looks “machine-like.” The trouble is that plenty of honest human writing is also smooth and predictable: formulaic reports, careful second-language English, anything that has been heavily edited.
1. People Cannot Reliably Tell
Confidence is not accuracy. A study in PNAS ran six experiments with 4,600 participants, who were asked whether self-descriptions such as dating and job profiles were written by a person or by AI. They could not tell.
What is more, people relied on shared rules of thumb that were wrong. First-person pronouns, contractions, and mentions of family made text seem human, whether or not it was. That makes people predictable, and anything predictable can be exploited. An AI system tuned to use those cues can come across as “more human than human.”
2. Detector Tools Make Costly Mistakes
The most cited warning comes from a 2023 Stanford study led by Weixin Liang. The team ran seven widely used detectors on 91 essays written by non-native English speakers for the TOEFL exam, and on 88 essays by US eighth graders.
The detectors were nearly perfect on the US student essays. On the TOEFL essays, the average false positive rate was 61.3%. Almost every essay, 97.8%, was flagged by at least one detector, and 19.8% were flagged by all seven. The writers’ more limited vocabulary and simpler sentence patterns looked “predictable” to the tools.

Even OpenAI could not solve this. It released its own classifier in January 2023, reporting that it caught only 26% of AI-written text while mislabeling 9% of human text, and withdrew it that July because of its low accuracy.
3. Detection Is Easy to Evade
The same Stanford study contained a twist. When the researchers asked ChatGPT to enrich the word choice of the TOEFL essays, the false positive rate dropped from 61.3% to 11.6%. In other words, the simplest way to avoid being flagged as AI was to use AI.
Other researchers have shown that paraphrasing tools can defeat a range of detectors, including some watermarking schemes. The practical result is uncomfortable. Detectors are most likely to catch careless or innocent users, and least likely to catch someone making a deliberate effort to deceive.
4. Watermarks Help, but Only When Built In
A watermark is a hidden statistical pattern added while the text is being generated. In a design proposed in 2023, the model is nudged toward a secret, shifting set of “green” words. The text reads normally, but anyone with the key can count green words and calculate how unlikely the pattern would be by chance.
Google DeepMind has taken this to scale with SynthID, describing its text watermark in Nature in 2024 and using it in its Gemini products. Watermarks give far lower false positive rates than guessing from style. Their limits are about coverage: the model’s provider has to add the watermark, heavy rewriting weakens it, and open models that anyone can run do not have to include one. In this interview, the researcher behind the 2023 design explains how it works.
5. Provenance Flips the Question
The most durable approach may be to stop hunting for fakes and start certifying originals. Content Credentials, built on the open C2PA standard, attach signed information to a file about where it came from and how it was edited. A camera, an editing app, or an AI generator can each add an entry.
Adoption is growing. According to a May 2026 report, Google is extending SynthID checks to Search and Chrome and adding Content Credentials to video captured on recent Pixel phones, with Instagram labeling that footage. Credentials are not a cure. They can be stripped, and a missing label proves nothing. But they give honest creators a way to show their work, which is something no detector can do. This short introduction from the W3C explains the idea.
What to Do Instead of Trusting a Score
If you teach, edit, hire, or moderate, these habits are safer than any detector.
- Never treat a detector score as proof. Use it, at most, as one reason to look closer.
- Be especially careful with second-language writers, who are the most likely to be wrongly flagged.
- Ask for process: drafts, notes, version history, or a short conversation about the work.
- Set clear rules for acceptable AI use, so that people can be honest about how they worked.
- For images and video, check for Content Credentials and trace the earliest source.
- When your own organization publishes AI-assisted content, label it plainly.
We discussed the leadership side of this in how misinformation reaches the boardroom and an earlier approach to verifying origin in blockchain and journalism.
Key Terms for Detecting AI-Generated Content
The conversation about detecting AI-generated content is full of jargon. These terms are the ones that matter.
- False positive. Human work wrongly flagged as AI. This is the error that harms innocent writers.
- False negative. AI-generated work that passes as human. This is the error that lets misuse through.
- Perplexity. A measure of how predictable a text is to a language model. Many detectors treat low perplexity as a sign of AI.
- Watermark. A hidden statistical pattern placed in content when it is generated, which can be checked later with a key.
- Provenance. A record of where a piece of content came from and how it was changed.
- Content Credentials. The public name for provenance information attached to files using the C2PA standard.
Any tool for detecting AI-generated content trades false positives against false negatives. Ask which error it was tuned to avoid, and whether detecting AI-generated content is even the right goal for your situation.
Quick Questions, Answered
Are AI detectors ever useful? They can help prioritize what to review at scale, for example in spam filtering. They are not reliable enough to accuse an individual.
Do watermarks solve the problem? They solve part of it for content from providers that use them. They do not cover every model, and they can be weakened by rewriting.
Is unlabeled content suspicious? No. Most content today carries no credentials at all. The absence of a label is not evidence of anything.
Final Thoughts: Verify the Source, Not the Style
Detecting AI-generated content by its style is a contest that detectors are slowly losing, and the people who pay for the errors are often the most vulnerable writers. The stronger path is provenance, clear norms, and human judgment about sources. That fits a wider pattern in how audiences now find information, which we explored in AI as the new gateway to news. Trust will depend less on guessing who wrote something and more on being able to check where it came from.



