What Happened
Substack has introduced an AI detection tool powered by Pangram, allowing readers to scan any post, note, reply, or comment longer than 100 words for an estimate of how much of the text may have been AI-generated. The feature rolled out across the web and iOS on July 21, with Android support coming soon.
CEO Chris Best introduced a new term for the problem — "Claudefishing" — defined as when readers "unwittingly invest their attention in something with no human thought on the other end." The tool is designed to surface an estimate rather than a definitive label, letting readers decide for themselves.
Creators can also scan their own drafts with Pangram and have the option to report inaccurate results. A new "How I make this" statement lets writers explain their writing process to readers.
The Accuracy Problem
The timing is awkward. Just last week, Epoch AI published research testing Pangram alongside GPTZero and Originality.ai against text written in imitation of a specific author's style, finding up to 18% of AI-generated passages went undetected, with scientific writing the most vulnerable category.
Chris Best acknowledged the limits honestly: "Pangram can only detect whether AI was used to make the text, not whether great human care went into creating it, nor whether AI tools were used as a source."
Why It Matters
Substack is navigating the same question every content platform faces: how to handle a flood of AI-written material without alienating legitimate writers who use AI as a tool. The framing as an "estimate" rather than a verdict is the responsible approach.
But the deeper structural problem remains: detection tools are trying to hit a target that moves faster than they can aim. Making an AI imitate a writing style takes one line of prompting, while detecting it requires distinguishing statistical patterns that get subtler with every model generation.
As Best put it: "Platforms that reward fakeness will create a race to the bottom." The question is whether detection tools — no matter how well-intentioned — can ever win that race.
