// writing

2026-07-10 // 15 min // ai, llms, work

not even wrong

An employee gets a message from her boss and can’t work out what it’s asking. She suspects it’s AI-written, so she pastes it into her own AI, which explains it and offers to draft the reply. Afterward she summed up the working relationship: “I can’t crack the code of working with [my boss], because it’s just his AI and my AI going back and forth.”¹

That’s one desk. The measured version of the same story comes from a survey of knowledge workers: the more people trust the model, the less they think about what passes through it, and the thinking that remains shifts to checking outputs.² That’s understanding, delegated at scale, from both ends at once.

The documents are multiplying, and everyone feels productive. What nobody can say anymore is who holds what the documents say.

the hive mind has a name

A company’s knowledge feels like a hive mind because it works like one, and the folk intuition has a 40-year-old scholarly name. Daniel Wegner called it transactive memory, and he found it first in couples: two people who each remember their own domain and reliably know what the other knows.³

It’s real and it’s trainable. Teams that learn a task together outperform teams of individually trained strangers, and the advantage comes from the shared encoding, not from familiarity or liking. But the system has joints, and the joints are exactly where documentation lives. Writing knowledge down only moves it if someone absorbs the reasoning on the other end: the people who helped produce a document read it the way it was meant, and the people who only received it mostly don’t.⁴ An artifact that never gets retrieved into a head is just storage.

One caveat, carried as a feature: nobody ever validated transactive memory at the scale of a whole company, even between humans.³ The loop is being installed into a system whose company-sized version was never shown to exist. And the loop routes around both joints at once: one model writes without a person internalizing, another model summarizes without a person decoding. The who-knows-what map keeps updating anyway. It increasingly points at nobody.

the producers and the bill

Who’s filling the pile? Two producers, and only one of them is the one you’re picturing. The first is the confident tourist. Dunning-Kruger survives its statistical critics only as a direction, and the direction is enough: the people with the least grip on a domain overestimate their standing in it the most.⁵ What tourists lacked before was throughput; producing a long, structured document used to require knowing something. Now the AI-era version of the effect has been measured directly: give people a chatbot on a reasoning task and the classic pattern vanishes, because everyone miscalibrates, gaining about three points of performance and about four points of self-assessment, with the most AI-fluent users the worst calibrated of all.⁶ The tool doesn’t democratize competence. It democratizes the feeling of it.

The second producer is underwater rather than overconfident. When the researchers who named workslop, AI-generated work content that masquerades as a contribution, went back months later to ask who makes it and why, the recipe they found was managerial: unclear AI mandates plus overwhelmed teams.⁷ The tourist and the professional ship the same fluent nothing for different reasons, and the receiver can’t tell them apart.

What receiving it costs has been measured properly only outside the office. The maintainer of curl watched confirmed vulnerability reports drop from better than 15% of submissions to under 5% when the LLM wave hit, and killed a bug bounty that had run six years; in all that time, the fire-and-forget AI channel produced zero real vulnerabilities, while the same tools in the hands of one accountable human produced roughly 150 fixed bugs. Clarkesworld closed submissions when machine-written stories hit 500 in a month against 700 human ones. And the Linux kernel’s arc is the one to memorize: this spring the AI-written reports turned genuinely good, and the maintainers’ review burden rose anyway, because the cost driver is volume through human judgment, not the quality of the output.⁸

Inside companies, the only number anyone has is a vendor survey in which workers self-report roughly two hours of cleanup per incident, published by a sponsor that sells the coaching to fix it.⁷ The extrapolation from bug trackers to meeting rooms is mine, and I’m making it.

Brandolini’s law, coined in a 2013 tweet and much older in spirit, prices refutation at an order of magnitude more energy than production. The half everyone forgets is social: refuting a colleague’s document means telling a colleague, sometimes a senior one, that their contribution is empty.⁹ Most people pay neither cost. In the workslop survey, recipients quietly rated senders as less capable and less trustworthy, and a third started avoiding them. Nobody flags the slop. It gets filed, and the who-knows-what map silently reroutes around one more person.

the best case

The case that the model belongs in this loop is real, and it deserves its full weight. Its best study put an assistant trained on top performers’ conversations onto a customer-support floor: productivity rose 15% on average, the newest agents gained most, and some of the gain persisted when the tool was off. That’s the optimistic version of everything here, the organization’s tacit knowledge extracted from its experts and dealt to its novices in real time.¹⁰ Consultants with GPT-4 did better work faster on tasks inside the model’s competence; professionals wrote 40% faster with output graded higher; and a meta-analysis across 106 experiments found real synergy on content-creation tasks whenever the human alone was better than the model alone.¹¹ On the right task, inside the frontier, with a person still judging, the loop adds. Conceded in full.

Now read the same studies for what they measured. The quality gains are clarity gains: Microsoft’s own controlled test found assisted emails rated 18% clearer by a blind panel, with no statistically significant difference in accuracy. Clearer, faster, prettier, and not one point more correct, in the sponsor’s own data.¹² The mechanism behind the leveling is compression: novices rise toward the house pattern, experts gain nothing or slip a little, and the collective diversity of ideas drops by about 11%. That’s redistribution of knowledge the organization already had, with nothing new entering the stock, and the learning that should mint the next experts doesn’t show up: model-assisted students scored best while the tool was on and learned no more than anyone else.¹³

And the boundary conditions from the meta-analysis read like a description of the documentation loop, written as a warning. The gains sit in content creation; judging what a colleague sent you is a decision task, the quadrant where human-plus-AI loses. The gains sit inside the model’s frontier; the consultants in the field experiment couldn’t tell where the frontier was, and outside it their accuracy fell off a cliff.¹¹ The gains require a human still validating before anything ships; that’s the node the loop automates away, on both ends. The best case for AI at work is a description of the loop with a person still in it.

what the dashboard says

The feeling arrives first. Output is the thing a dashboard can see, and output explodes: assisted emails run 167% longer than human ones without carrying more, and in the study that bothered to check, most people who’d just written an essay with a model couldn’t quote a line from it minutes later.¹⁴ The cost lands elsewhere, on the receiving calendar, two unlogged hours at a time, visible on no chart. Volume up, comprehension unmeasured: the quarter reads as a good quarter.

Then the knowledge stock stalls while the document count climbs. Clarity without accuracy, leveling without learning, diversity down. And the alarm that should trip is wired to the wrong signal, because offloading inflates your sense of what you know more reliably than it changes what you know: searching makes people feel smarter about things they never looked up, and across the replication literature, that self-perception effect is the one that holds while the memory effects wobble.¹⁵ The heads empty quietly, feeling full the whole way.

Here’s the strange part. The instrument for checking exists. A corpus method published last year detects LLM-assisted writing at population scale with under 3.3% error, and it’s been pointed at public text for two years: by late 2024, machine-assisted shares ran from about 10% of job postings to 24% of corporate press releases, and had stopped climbing.¹⁶ It has never been pointed inside a company. There is no published measurement, none, of how much of any organization’s internal writing is machine-made, or of what the people who filed it still know. The loop closes exactly where nobody’s looking, and the feeling of productivity persists because nothing is positioned to contradict it. The web’s version of this loop at least has a market selling fresh human data back into the pipeline. A company’s only fresh supply is its own people’s attention, and that’s being spent on triage.

A lot of my job is reading what models wrote, mine and other people’s, so take this last part as testimony rather than measurement. The documents I’ve learned to dread aren’t the wrong ones. Wrong is workable: a wrong plan makes a claim, a claim can be tested, and the test teaches somebody something. The kind I dread is fluent, long, structured, confident, dense with decisions, and after an afternoon inside one I still can’t tell you what problem it’s trying to solve, which alternatives were considered, or which of its choices a person actually made. There’s nothing to refute. There’s fog to clear, and fog-clearing is invisible work that costs real hours and makes you the difficult one for asking what the pages are for.

Physics keeps an old insult for work like that, handed down through a colleague’s memoir of Pauli: not even wrong.¹⁷ Too unmoored to argue with, which is worse than mistaken, because argument is how a group of people comes to know anything. Everything is documented now. Ask the dashboard who understands it.