Take Nobody’s Word

Last week Anthropic published a measurement of judgment. The setup: take real research sessions, find the moments where a scientist chose what to do next, ask a model to make the same call, and have a second model, shown how the session eventually played out, judge which call was better. The skill has a name in the field, research taste, and now it has a trend line.

I work inside a discipline that's been told its knowledge doesn't count because it can't produce that chart. Design's claims are qualitative: that hierarchy changes comprehension, that naming shapes behavior, that a system's coherence decides whether the work survives the team that built it. A working designer knows these the way a mechanic knows what that ticking sound is. The institutional response never changes: "That's subjective." "Where's the data?" The data was there all along, a career of experience compressed into what's more often called intuition, or taste: subjective knowledge no instrument could read. Illegible, not absent.

In 1665, Robert Hooke published Micrographia, a book of drawings made at a microscope. The instrument was not new. It had existed for half a century and changed almost nothing, because looking is not the same as showing. Hooke, the Royal Society's curator of experiments, drew what he saw: the chambers in a slice of cork, which he named cells; a flea across a fold-out plate, armored like siege equipment. Before the book, claims about the microscopic world were testimony and the Royal Society's own motto was Nullius in verba (take nobody's word for it). The drawings ended the argument.

The same arc is running right now inside AI research. Neural networks are built from millions of small units called neurons, the name borrowed from the brain, and for most of the field's history nobody could see what any single one of them was doing. Interpretability is the discipline that observes. For years it was dismissed inside its own field as too qualitative to be science: researchers zooming into networks, publishing images of what they found. Chris Olah saw the Hooke parallel first; his 2020 introduction to the field closes on Micrographia and the point that the discovery of cells was a qualitative result. His teams kept publishing anyway: images of single neurons that fire for one concept however it arrives, as a photograph, as an illustration, as written words. The images accumulated until the structure was impossible to deny. The methods, built first on vision models, now run inside the language models everyone uses, where researchers have started using them to check what a model is actually doing instead of taking its word for it.

Disciplines get dismissed for being illegible, not necessarily for being wrong. The structure design describes is real the way the cells in the cork were real before 1665. What changes is whether anyone can show it. The measurement Anthropic published is that moment arriving for judgment itself: taste benchmarked, the next-step call scored. The method isn't specific to research. Any call that produces an outcome can be judged against the outcome, and design is nothing but calls that produce outcomes. Expect the same revision: a century of being told it was decoration, then told it was a science of structure all along, as if the knowledge began when the measuring did.