13 Oct 2026 / Essay

A score is not a record

Since 21 July 2026, as reported by Startup Fortune, a reader on Substack has been able to run a Pangram scan on a writer's post and get a number back. The number arrived in the middle of an argument it did not start and cannot finish: parts of the platform's own community called the feature a witch hunt, as reported by 404 Media, and the split it opened was still being covered by NPR in August. We have argued here before that what a number cannot see, the work stops doing. A score is cheap to count. A record is hard to earn.

Look at what sits behind that number, and the picture is considerably more careful than the number implies. On 26 September 2026, AFP reported that different detectors scored excerpts of one disputed novel anywhere from "very likely" human to "100%" AI. In the same report, AFP relays The Atlantic's description of Pangram's chief executive as calling the tool a starting point for an investigation rather than the arbiter of one. Substack's own announcement of the feature, as reported by Startup Fortune, said the tool cannot judge whether human care went into a piece.

That is a remarkable set of caveats to hang on a number that arrives with none of them attached. If the score is a lead and not a finding, where should verification actually live?

What a detector can establish, and what it cannot

Start with the strongest version of the other side, because the weak version is a straw man. These tools are not snake oil. Pangram's chief executive put its false-positive rate at roughly one in ten thousand, as reported by 404 Media. A University of Chicago paper from October 2025 found near-zero error rates for Pangram on medium-to-long excerpts, as reported by AFP. Take both at face value, as I do, and the argument that follows does not weaken by a word.

Because the objection that matters is not accuracy. It is category.

A detector is handed finished text and returns a probability about that text: how closely does this prose resemble what a model produces? That is a real question, and a well-built tool can answer it well. It is simply not the question anyone is asking when a dispute begins.

The question in a dispute is: who wrote this, and how? Was there a draft? Did a person conceive the argument and a tool tighten the sentences, or the reverse? Did someone paste in a chatbot's paragraphs? Did an editor, hired by the author, run the manuscript through something without telling anyone? Those are facts about a process: events that happened in order, to particular people, at particular times. A process leaves records when someone chooses to keep them. It does not leave them in the finished prose.

This is why better accuracy does not dissolve the problem. Push the tool to its limit and imagine a detector that is right every single time about whether a model generated a passage. It still returns a probability about text, not a record of process. It cannot tell you whether the writer directed that model, whether those sentences are the author's own reworked three times over, or which hand in a chain of hands introduced them. The number would be correct and the dispute would remain open. That gap is structural. It is not a defect waiting on a better model.

There is a second asymmetry worth naming. A score lands on a person, and the person is then asked to disprove it. But the evidence that would disprove it — drafts, version history, correspondence, the plain sequence of who did what and when — is exactly the material that a writing process does not keep unless it was built to keep it. So the accused is asked to produce a record that the work was never designed to create. Some writers happen to have one, because a tool they use kept it by accident. Others do not, and the absence is then read, by an audience that would like the matter closed, as something uncomfortably close to an admission. A system that puts the burden of proof on whoever is least equipped to discharge it will produce bad outcomes even when its measurements are excellent.

What the consequences have been

The gap is not abstract. It has produced disputes that were expensive for the people inside them, and that no score could close.

Consider the novel Shy Girl. As reported by Startup Fortune, relaying The New York Times, Hachette cancelled the book's US release and said it would discontinue the UK edition, which had been published in November 2025. The same Startup Fortune report carries the author's denial: they did not write it with AI, and an acquaintance they had hired to edit it used AI without their knowledge. Notice the shape of that. It is a contested case, not a settled one. Nothing in the reporting I have read calls the flag false. The thing that would settle it is not a sharper score. It is a record of who touched the manuscript, when, and with what.

Consider a Commonwealth Short Story Prize winner whose story, as the Guardian reported in May 2026, was flagged at 100%. That flag reached the public, in that same report, through a researcher's post on Bluesky citing Pangram, rather than through the prize or the magazine. As reported by Startup Fortune, the Foundation reviewed his drafts and stood by him. The writer's own position, as reported by Slate, is that he uses AI for research only, a defence of AI-as-tool rather than a blanket denial of ever opening one. Here again, what moved the case toward resolution was process evidence, not a sharper reading of the text.

Granta subsequently withdrew from external publishing partnerships and, as the Guardian reported in June 2026, gave its own stated reason: for the sake of its own editorial integrity. The Guardian's report ties that withdrawal to the AI controversy, and Granta's statement names the controversy as the context it was deciding in. What the statement does not contain is a finding about any particular story.

And there is one case that a source calls an error outright. As reported by Startup Fortune, relaying The Atlantic, the writer Taylor Lorenz was accused on X of publishing AI-generated text, denied it and had the edit history — and Pangram's own co-founder found that the tool had erred.

Set beside these the finding I opened with: as AFP reported, excerpts of a single disputed novel run through different detectors produced results ranging from "very likely" human to "100%" AI. What does that leave an ordinary reader holding? Not a close call between two hesitant instruments, but two confident instruments pointing in opposite directions, and no procedure available for deciding between them. Accuracy figures are measured in aggregate, across corpora, under conditions a laboratory controls. A dispute is always one case, about one person, and the person inside it does not get the aggregate.

Those instances are not one kind of thing. One was reported as an error. Others, on the reporting above, are contested still. One is a withdrawal that followed, for reasons its publisher stated itself. They share a single feature. Wherever anything got closer to settled, it got there through records of how the work was made: the reporting shows drafts reviewed, edit history produced, a vendor checking its own output. The score opened the argument. In none of these did it end one.

The alternative is a design choice

So the real question is not which detector to trust. It is this: what would it take for the process facts to already exist, in a form nobody has to reconstruct under accusation, after publication, by strangers?

Pagecut's answer is a decision about how the production line is built rather than about which tool to buy. We designed the line so that fact-checking is a distinct stage before publication, with the actor doing the checking separate from both the drafter and the editor. That separation is the whole point of the stage. A check performed by the person who wrote the thing, or by the person who shaped it, is not an independent check; it is the same judgement asked twice.

It is worth being concrete about the cost of such a stage, because the cost is why this is a choice rather than a default. It is an additional pass before anything goes out, performed by someone who has to read the piece properly to check it at all, and it is specified for every piece we publish from here on, including, and especially, the uncontroversial ones that nobody is ever going to question. That is the bargain, and it only means anything if it is unconditional. A control that runs only once you already suspect trouble is not a control. It is a reaction, and a reaction arrives in the same place a detector does: afterwards, reconstructing.

Two things have to be said plainly about what this does and does not offer.

First, the fact-check gate checks claims, not authorship. It asks whether what a piece asserts is true and sourced. It does not establish who wrote the piece, and nothing in the design should be read as though it did. Anyone claiming their internal gate settles authorship has simply swapped one unearned verdict for another.

Second, what the design does offer on authorship is narrower, and I think more honest: declared, role-level provenance. Who held which role, whether drafter, editor or checker, is declared where the work happens, rather than inferred from the output afterwards by someone who was not there. It is a record made at production time, by the people doing the production, as a condition of the work moving forward at all. It is not a score, and it does not pretend to be one.

A third thing, about the status of every sentence in this essay that describes Pagecut's own line. All of it describes design intent: how the line is specified to work. None of it is a report that the line ran, or caught something, or performed well. Those are different claims, they require different evidence, and this essay does not make them. The restraint is not modesty for its own sake. It is the same discipline the essay is arguing for: a claim about process needs a record rather than an assertion, and that applies first to our claims about our own process.

Choosing before the accusation

Detection-after-the-fact asks a finished artifact to confess how it was made. It performs well in controlled tests, and the caveat quoted near the top of this essay is the vendor's own: a starting point for an investigation, not the arbiter of one. But its ceiling is fixed by what it is: inference about text. No amount of engineering moves that ceiling, because the thing being asked for was never in the text.

Verification-by-construction asks something different: that the process declare itself while it is happening, in the place where it happens, by the people accountable for it. It costs more, and it has to be paid for during the long stretch when nobody is accusing anyone of anything. It is also unglamorous: no number, no dashboard, no moment of revelation. Just a stage specified so that a piece does not go out until a named someone who did not write it has checked what it claims.

One of these produces a number a reader can argue with. The other produces a record a reader can check. Neither is magic, and neither makes writing honest on its own. A determined fabricator defeats both. The difference is what survives contact with a dispute. A score is a probability about a text. A record is a record. When an accusation arrives — and on the evidence above, it does arrive — only one of those is something you can put in front of someone.

That is a design choice, not a technical finding, and it should be argued as one. We would rather make it before the accusation than after it. A score is not a record, and accuracy will never make it one.