// writing

2026-07-04 // 11 min // ai, llms, data

eating its own tail

There’s a Python library called wordfreq that tells you how often humans use any given word, in dozens of languages, compiled from big samples of real usage. In 2024 its maintainer, Robyn Speer, froze it at 2021 and announced there would be no more updates. Her stated reason: “I don’t think anyone has reliable information about post-2021 language usage by humans.”¹

She wasn’t early. By one of the few defensible measurements, AI-written articles overtook human-written ones in November 2024, and the two have run roughly even since.² The humans still writing are drifting too: machine-preferred vocabulary is up in 26 of 34 languages in news prose, and rising in unscripted speech.³

Now notice where the models get their training data. They write the web, and they eat the web. The snake found its tail.

I build on these systems for a living, so I’ve been following the argument about what happens next. It has a name, model collapse, and two camps that mostly shout past each other. Everyone’s asking whether the models will get worse. Almost nobody’s asking what goes missing first.

the tails go first

The mechanism is peer-reviewed and boring in the way settled things are boring. Train models on their predecessors’ output, indiscriminately, generation after generation, and you get irreversible defects with a consistent signature: the tails of the original distribution disappear first.⁴ The rare, the particular, and the weird go before the common does. It doesn’t take much, small synthetic fractions degrade the scaling laws and more data doesn’t rescue them. And it doesn’t look like anything: factual accuracy decays while fluency holds, so the output stays confident and well-formed as it goes wrong.⁵

“Model collapse” has come to mean eight different things in the literature, which is a lot of why the public fights go nowhere.⁶ The version that died, models dissolving into visible gibberish, you can forget. The version that matters gets measured least:

The early collapse demos produced failure you could screenshot. Coverage collapse produces prose you can’t fault, missing things you didn’t know to ask for. The injury is an absence, and absences don’t show up in screenshots.

the escape routes

The strong case against the doom is real, so here it is. Collapse in those experiments comes from replacing the training data wholesale each generation. Accumulate instead, keeping the old real data in the mix, and test error is provably bounded. Filter synthetic data through even an imperfect verifier and collapse doesn’t come. Microsoft’s phi-4 trained synthetic-heavy on purpose and beat its own teacher; Meta trained more than a thousand models comparing recipes and found rephrased synthetic data at a one-third mix just works.⁷ It’s good work. The doom version of collapse really is dead.

But look at what every escape route has in common. Accumulation keeps real human data in the mix forever. Verification needs a verifier grounded in something. The strongest result in the family, a proof that self-verification prevents collapse “even in fully synthetic training regimes”, anchors everything to a frozen copy of the initial model, the one trained on real data, and excludes outputs that drift from the original data manifold.⁸ The route that advertises independence from human data is an embalmed sample of human data running quality control forever. Whatever that sample didn’t cover, no generation of the process will ever recover.

And none of them measures the tails. Test loss, task accuracy, ROUGE: means, all the way down, in all four route-defining papers. The closest measurement anyone has made points the wrong way for comfort, because quality filtering makes diversity decline worse: “quality filters typically favor more common and less inventive samples.”⁹ The field’s own position paper says it flat: real tail data will be lost, it’s unclear how much or how fast, and “we strongly encourage more research.”⁶

The debate has been arguing about the mean. The tails are the stakes. As I write this, nobody has measured whether the escape routes preserve them.

laced at the source

The escape routes run on fresh human data. So it matters that fresh human data is now an industry, and it matters more that the industry is broken in exactly the wrong way.

The old supply is finite, closing, and priced: the standing forecast has public human text exhausted within years, 5% of a standard training corpus went fully restricted in a single year, a settlement just priced authors’ books for training at $3,100 a work, and Reddit rents its archive to Google for about $60 million a year.¹⁰ So fresh human thought gets made to order instead. Data annotation cleared roughly $3.5 to 4 billion across its six biggest vendors in 2025, and what the money buys has climbed: the Kenyan workers who labeled ChatGPT’s worst content took home under $2 an hour, while Mercor’s thirty thousand vetted scientists, doctors, and lawyers now average over $85, on a platform paying out $1.5 million a day.¹¹

Cheap instruction data is already synthetic; what can’t be generated is expert judgment. “If companies can buy more GPUs instead of paying for data, they will do that,” writes Nathan Lambert, who builds post-training pipelines.¹² They keep paying. Meta paid $14.3 billion for 49% of the anchor vendor. The market is not buying human data in general. It is buying the tails.

Now the broken part. The humans paid to produce human data use the models: 33 to 46% of crowd workers on a task where researchers could check by keystroke, 34% of survey participants on a major research platform, and the assisted answers come out measurably more homogeneous, tail-flattening inside the human pipeline itself. Countermeasures cut it roughly in half, no better, and the compliant workers score worse on the surface metrics than the cheaters, so every incentive points toward tolerating the lacing.¹³

The documented worst case ran for eleven months at the very top of the market: Scale AI’s program for Google, training what became Gemini, flooded with what the company’s own paperwork called spam, “GPT-generated thought processes”, staffed without background checks, and patched by running the expert data through an AI detector to check the humans weren’t machines. That was the quality control on the fucking escape route.¹⁴ Below all of it sits a $70 black market in verified annotator identities.¹⁵

And the contamination rate inside the premium pipelines, the ones feeding frontier post-training? Never measured. Same shape as the tail question: measured in the adjacent settings, unmeasured exactly where the stakes are, and the industry’s optimism resting on the unmeasured part, both times.

what the model chooses

So the web sits at parity and drifting, the models eat the web, the tails go first, and the escape routes all run through a human supply that’s already machine-laced. What’s left is the question of trust: can the models be counted on to steer away from the average on their own?

I can answer that one from my desk, as testimony rather than measurement. Supervising LLMs is most of my working day now, and the substance of it is exactly this steering. When I plan work for a model to implement, the planning is largely about keeping it away from the average: pinning the choices where the average answer would be wrong. And when I leave decisions open, I check which way they got made. They never come back made toward the tail. Not once. The model picks the average, every time, because the average is what it is.

The tails are people: the physician writing edge cases by the hour, the maintainer of the weird library that never made a benchmark, the writer whose register no model quite has. I build with these models daily and want them good, which is why their supply chain worries me, and everything on this site is written and signed by a person, sitting in the crawl. A rounding error of tail, kept on purpose.

Speer stopped counting because the count had stopped meaning anything. The escape routes assume the humans keep writing, the annotators stay human, and the rare stays in the data, and nobody is measuring any of the three. The models are about to find out what else stopped being counted.