The Spec Is Where You Think

On January 8, 2026, Cloudflare made a tiny change. When you ask a DNS server "where does this domain live?", the answer comes back with several pieces of data inside it, in a list. Cloudflare changed the order of that list: it moved some data that used to come first to the end. It was an optimization to use less memory.

For the next 2 and a half hours, half the internet broke.

The function every Linux program uses to turn a domain into an address took for granted that this data came first. Nobody had ever told it that it had to come first. It assumed. 3 models of Cisco switches ended up rebooting themselves, in a loop.

The interesting part isn't the bug. It's the explanation Cloudflare gave afterward:

"The problem is that the RFC primarily discusses ordering in the context of RRsets but doesn't specify the ordering of different RRsets relative to each other within a message section. This is where the ambiguity lives."

That document is from 1987. It said the answer came "possibly prefaced by" certain data. "Possibly." It didn't say "must," it didn't say "should." For almost 40 years every programmer who read that filled the gap with whatever seemed right, and it worked, because by coincidence everyone filled it the same way.

A case nobody specified isn't a case that doesn't happen. It's a case where everyone decides on their own, and everything works as long as those decisions happen to line up.

What Cloudflare did next is the best part. They reverted the change and then wrote the document that was missing: a formal proposal so the ordering is defined once and for all.

Why this matters now

There's an idea going around that SDD and BDD are the new way of working. I don't fully buy it. BDD is 20 years old, and writing specs before coding was called waterfall when it went wrong.

What did change is smaller and more uncomfortable: writing code got cheap, and deciding what should happen didn't.

Typing used to be the slow part. Doubts and gaps were paid off slowly and late: they showed up when someone reviewed the code, when QA tested it, sometimes once it was already in production. Expensive, but in installments. Today an agent builds an entire screen in 25 minutes. The gaps didn't disappear, they all piled up in one place. Now they get paid all at once, at the end, when there's already code written with total confidence on top of something nobody checked.

My conclusion isn't that we need more documentation. It's something else:

The spec isn't the paperwork you do before the work. It's where you think.

(When I say "spec" I mean the document where you write down what something has to do before you build it. Nothing more than that.)

The Cloudflare thing happened in a document thousands of people read over decades. Below are two cases of my own, much smaller, that are the same thing.

Case 1: the seat someone else bought while you were looking at it

A movie ticketing platform. The screen is the seat picker: you choose a showtime, see the map of the room, tap seats, move on to payment.

Sounds like "drawing a grid." Written out, it had 22 acceptance criteria. An acceptance criterion is a sentence describing something that has to happen, written so you can check afterward whether it happened or not. There were 22 because a grid of seats isn't a grid: it's a pile of possible states over something several people want at the same time.

StateWhat happens
Just landedmap loaded, the continue button is off
Loadingyou see the outline of the screen (a skeleton), never half a map
Something failederror message and a button to retry
Picked between 1 and 9summary grouped by type, continue button on
Tried for number 10it won't let you, and it explains why
Conflictthe seat you picked was bought by someone else while you sat there
Came back from paymentyour selection is still there

That second-to-last one is the interesting one. The screen asks the server every 20 seconds which seats are still free (polling). Between one question and the next, someone in another city buys the one you had selected. What does the user see?

The acceptance criterion, exactly as it was written:

Criterion 14: The map polls inventory while the user is on the screen. If they select a seat that was sold since the last poll, they're told, and the map refreshes so they can pick another.

And then the same text, but turned into a test that runs on its own:

Given the last poll has not yet reflected that the seat was sold
When the user selects it and the next poll reveals it is gone
Then a message is shown and the map refreshes
And the seat leaves their selection and the total

That's called Gherkin, and it's simply writing "given this, when that happens, then this other thing." The point is that the product person understands it, the testing person understands it, and a machine understands it.

That case isn't in any design. You can't see it. A design shows you what things look like when everything goes right; the states nobody drew are still going to be decided by someone, and if you don't decide them by writing them down, they get decided by whoever is coding at seven in the evening.

Each of those cases has a note about what verifies it: whether it's a standalone function (a unit test), simulating the screen without a browser (a component test), or opening a real browser and driving it with Playwright (an E2E test). An acceptance criterion without that is a wish.

If you want to see this same problem in code you can open right now, Sylius is an open-source online store with 855 files of these scenarios. Several model the same thing under another name: units "on hold" versus units "on hand," what happens when someone cancels and the product goes back to stock, and this one, which is my seat conflict wearing different clothes:

Given I added a t-shirt to the cart
When I have proceeded through checkout process
But the t-shirt has been disabled
When I try to confirm my order
Then I should be informed that this product has been disabled

That "But" in the middle, marking that the world changed while the user was walking through, is the whole idea.

What showed up before a single line of code

This part changed my mind about what writing specs is for.

Before coding, we went and asked staging what it actually returned, and compared that against every criterion. Three things showed up.

First: the approved design had the math wrong. The signed document said the savings with the cinema card was the price difference on one seat multiplied by the number of seats. But in a room with regular and premium seats mixed together, each type saves a different amount: one saves $5.000 and the other $6.000. The correct math is adding up the savings for each seat, one by one. With two seats of different types, the approved design showed a false number.

Nobody lied. The math works perfectly as long as all the seats are the same, which is how the example was drawn. Same as the DNS document working for 40 years.

Second: the service fee field (booking_fee) came back empty. The prototype had "$1.600" hardcoded and nobody knew where that number came from. It became a question with a name attached to whoever had to answer it, instead of a magic number that survives three months because it works.

Third: a seat's state could be anything. The documentation said a seat was either "available" or "sold." The real server returned "available" and "broken." If the code only understands the closed list of options it knew about (a closed enum), the day a third one shows up it breaks, or worse, the seat disappears from the map without anyone noticing. It got written like this: any state that isn't "available" renders as unavailable, even one that doesn't exist yet (open enum, with a fallback).

None of those three show up while coding. They show up when you write what has to happen and then go check whether the real world agrees.

Case 2: the number that looked like the same number

Another project, another language, another kind of finding. An app for pooling money between friends to buy a gift.

A ticket had been stuck for over a month: "delete confirmation for things that already received money." The real question, the one nobody had answered, was a single one: if someone already put money in, what do we do when the owner wants to delete it? Do we archive it, block it, or hide it?

As long as that question went unanswered, the ticket couldn't be estimated, built, or closed. And it isn't a hard question. It's a question with no owner.

When product answered "we block it," writing the spec was what showed why the other two options were far more expensive than they looked. "Archive" sounds like a different button in a modal. Written out, it was: the "archived" state doesn't exist in the database, it had been deliberately removed a few months earlier, and it opens 4 questions nobody answered. Where do archived things show up? Who can un-archive them? Do they still count toward the totals? Do they detach from the events they were attached to?

None of those 4 is decided by someone on frontend. That's the difference between estimating 2 days and estimating 2 weeks.

My favorite finding

In that same spec there's a section titled, verbatim: "the real trap: raised here is NOT the deletion rule."

The app shows on screen how much money an event raised. And it won't let you delete an event that received money. They look like the same thing. They aren't.

The number on screen adds up the money that arrived through the goals currently attached to that event. The block looks at something else: every donation pointing at the event, attached or not. If someone donated through a goal that was later detached, that money still points at the event and still prevents you from deleting it, but it no longer shows up in the number you see.

So: if you build the confirmation dialog by reading the number on screen (which is the obvious thing, and it's the data already sitting right there), you tell the user "sure, go ahead, delete it" on an event the server is going to refuse.

You don't find that by looking at the screen or reading the ticket. You find it when you write "the dialog branches on whether it has money or not" and force yourself to answer where that yes-or-no comes from. That's when you discover there are two sources that give the same answer 95% of the time. Same as DNS.

Deciding is the new work

Both cases are the same thing seen from two sides: the hard part wasn't building, it was deciding. And deciding well takes research, not intuition.

The seat spec has 10 facts that came from going and asking the real server, not from reading the documentation. The docs said seats came in a list; in reality they came in rows, and each row had its seats inside. The prices with and without the card weren't two fields, they were two separate lists you had to match by an id (a join). And the money came in cents, so the number 3420000 meant $34.200, not three and a half million.

None of that is in the ticket. All of it changes the code.

There's a measured reason to distrust documentation here. AI models handle well-known public tools reasonably well, because they've seen them a thousand times. But with the internal parts of your own project they make things up far more often, simply because they've never seen them. What's yours and only yours is what you can least take for granted.

Research isn't only technical either. When a state isn't drawn, you have to go find out what the old system did, what the competition does, what the user expects. On the seat screen, the "someone took it" notice had no approved design, because that case didn't exist in the prototype. It had to be decided. It started as a soft 3-second toast, it was tested and turned out to go unnoticed, and it ended up as an alert that names which seat you lost ("Seat K18 is no longer available"), with a close button, 7 seconds on screen. It's still written down as an open question for design, with screenshots.

That's the honest way to close an ambiguity: decide with judgment, write down that it was your decision and not an approved one, and name who has to confirm it.

Don't delegate your thinking

Here's the rule that organizes everything else.

You can delegate writing the code. You can delegate a good chunk of the review. You can even delegate the first draft of the spec. What you can't delegate is deciding what has to happen, because that isn't a task: it's judgment, and judgment is the one thing that didn't get cheaper.

That's why my flow has an explicit stop: the agent writes the spec and halts there. A human reviews it before a single line of code gets written.

This isn't bureaucracy. There's a very good critique of all this, an article bluntly titled "Spec-Driven Development: The Waterfall Strikes Back", which describes how the agent marked the "verify implementation" task as done without having written a single test. That critique doesn't refute the practice, it confirms it. The agent ticked the box, nobody verified. If your process depends on the agent grading itself, you don't have a process.

The second thing is that the agent has to ask instead of assume. This is measured: in a benchmark with real tasks pulled from GitHub, an agent given incomplete instructions solves 54.8% of cases. The same agent, when it notices it doesn't understand something and asks, goes up to 69.4%. The ceiling, handing it all the information up front, is 70.8%. So: asking recovers almost everything you lose by working blind.

And there's an odd detail in that same research. Making the model "think harder" improves code quality, but barely improves its ability to notice that something is ambiguous. Those are two different skills. Thinking harder doesn't save you from assuming wrong. You have to explicitly ask it to ask.

I've seen this fail on a small scale. An agent built a whole screen very well, and had one word wrong: it said "Reached" where it should have said "Completed." It wasn't the agent's mistake, it did exactly what the ticket said. The change had been discussed in a meeting and never written down. What isn't in the spec doesn't exist, and now "doesn't exist" turns into code in 25 minutes.

The third thing is cross-review. I use 2 automated code reviewers that look at the same code without knowing what the other one said:

One clarification is worth making, because this practice has a bad reputation. What's measured as failing is something else: many copies of the same model, with no assigned roles, talking until they vote on an answer. That's where the known problems show up, and the worst one is that the agents abandon what they had correctly worked out in order to copy what the majority says. The problem is the consensus, not the discussion. If your system treats disagreement as an alarm instead of averaging it away, that can't happen to you.

Even so, the last gate is never a code reviewer. A model judging code changes its mind if you change the variable names. A model's opinion isn't proof. Running the code is.

What each net catches

With code being cheap, the work moved to both ends: decide before, verify after. Here's the rundown of the second half, with what each net catches and what slips through the holes.

The spec (what people now call SDD) catches contradictions between sources: the design against the ticket, the ticket against what the server actually returns. What slips through: everything nobody thought to ask about. Honest disclaimer: no serious study exists showing that working this way improves outcomes. What exists is several large companies formalizing it, and one coherent critique. I use it because it found me concrete bugs, not because it's proven.

The "given, when, then" scenarios (that's BDD) catch the ambiguity of language: they force you to name the state you start in, what happens, and what you should be able to see afterward. The format is from 2006, from Dan North, who invented BDD, and his original example was an ATM: given the account is overdrawn, when the customer requests cash, then the screen rejects the request and returns the card. Worth noting that the person who invented BDD picked an ATM to explain it, and the documentation of the most-used tool picked an example about cucumbers. That explains a fair amount about why BDD struggled to catch on. What slips through: they're expensive to maintain, because every sentence you write needs someone to program what it means. That's still true today. And on whether they help an AI agent understand better: it sounds plausible, but I couldn't find anyone who has measured it.

Unit tests, the ones that catch a single function, catch the arithmetic bugs: the savings added up wrong, the grouping by type, the money formatting. What slips through: everything that only shows up when the pieces come together. They're cheap and fast, and that's why they're still the base.

E2E tests, which open a real browser, catch what only exists when everything runs together: navigating, going back, not losing what you picked, the whole flow. What slips through: the fine detail of the math, and they're slow and break for any reason. On the seat project the real ratio is 174 unit tests to 15 E2E. People say a lot that this ratio flipped with AI, because the agent produces whole screens instead of standalone functions. I couldn't find a single data point supporting that, and my own project says the opposite.

Then there's mutation testing, which almost nobody runs and which makes more sense now than ever: instead of reviewing your code, it reviews your tests. It deliberately introduces a bug into the code and checks whether any test complains. If none complains, that test wasn't verifying anything. Meta built a system like this across more than 10,000 files, and the person running it explained it better than I can: the percentage of code covered by tests (coverage) can still fail to detect a bug, because a line running doesn't mean anyone is watching what it did.

And finally there's monitoring, watching what happens in production, which catches the one thing none of the others can: real people, using the app in ways nobody imagined, with data nobody made up. This year's industry data shows something uncomfortable: the more AI gets used, the faster things ship and the more things break, both at once. That doesn't get fixed by writing better. It gets fixed by watching.

I'm not the only one saying this is the list that matters. Robert Martin, better known as Uncle Bob, who wrote Clean Code and much of what we learned about testing, has spent months building a system where agents are boxed in by tools that force them to follow a process. His list is nearly word for word the one above: Gherkin, code, unit tests, quality metrics, mutation testing. His argument is the same as mine by another route: long instructions in a prompt get ignored as the conversation grows, and a tool that runs on its own doesn't.

And a few months later he wrote something even more honest, worth quoting in full because it's the trap in all of this: "I've been pushing very hard on overloading with tests. Gherkin test unit test QA test mutation test gherkin mutation test. It's easy to make the AI's do these things. But just because we can do them doesn't mean we actually should." He's right. None of these nets is free, and putting all of them everywhere is its own way of not thinking.

The test that couldn't fail

The code reviewers found two bugs the spec didn't prevent.

One: the service fee took the most expensive one across all zones in the room and multiplied it by the number of seats. If you bought one regular and one premium seat, both paid the more expensive fee. The correct data, the per-zone one, was stored and never read. Two: when someone bought your seat out from under you, it left the notification toast but not the cart. It stayed selected, it kept adding to the total, and it could no longer be touched.

These are implementation bugs, not specification bugs. But look at what didn't happen when they turned up.

The service fee is wrong by a small amount of money. It's the kind of thing where someone asks "wait, wasn't it like this?", someone else digs up the document, a third person remembers it worked differently in the old system, and 40 minutes and a Slack thread go by before anyone touches a line. That conversation didn't happen. Criterion 17 already said, from before the code existed, that the fee is charged per seat. So the finding didn't come in as "hey, this looks off to me" but as "this violates criterion 17."

When the correct behavior is written down and numbered beforehand, the finding classifies itself. It stops being the reviewer's opinion against the coder's opinion, and becomes a comparison against a text they both accepted. What you save isn't the bug, it's the argument about whether it's a bug. The proof by contrast is the conflict notice: that one had no written criterion, and it's still unresolved.

And then something worse than the bug showed up. There was a test that lied.

The booking fee test used the same value for every zone. And since the bug was precisely "it applies a single value to every zone," the test went green with the broken code and with the healthy code. It couldn't tell them apart. Fixing it wasn't correcting the math: it was changing the test so the zones had different values. Only then could it fail.

That has a name and it's measured. If you ask a model to write tests while looking at code that already has a bug in it, the model understands that broken behavior as the intended behavior, and writes a test that blesses it: green on the broken code and red on the fixed one. In a measurement across 11 different models and real bugs, that kind of test shows up 8 times more often when the model looks at the buggy code than when it looks at the healthy code. And the tests that actually work drop to 1/3. What improves it, measured? Giving it the spec: telling it what has to happen, instead of letting it infer that from the code.

A test that can't fail isn't testing anything. And an agent that writes the tests by looking at the code, instead of looking at what was decided, produces exactly that, at scale.

That's why the last stretch grew. The sold-seat conflict was verified against the real system: actual seats were bought from another session, the seat was watched turning grey within the 20 seconds, and the selection was confirmed emptying with the total back to zero. 3 times, to be sure.

This explains the noise around AI. The problem isn't code that fails obviously, that shows up right away. The problem is code that's almost correct. 2 out of 3 developers say their biggest frustration is answers that are "almost right, but not quite," and nearly half say fixing generated code takes them longer than writing it from scratch. Every year AI gets used more and trusted less.

The point

In 1999 NASA lost a probe on its way to Mars because one program delivered numbers in pounds and the other expected newtons. What almost nobody mentions is that the spec existed and was correct. There was a document, a software interface specification, defining the format and units of the data one program handed the other: the same thing as the data contract I went and verified against staging in the seat case, narrower in scope. It said Newton-seconds, in so many words.

Nobody followed it. And the part of the report I find most interesting is that NASA doesn't file this as a units error: contributing cause number 8 is called "verification and validation process," and it says the specification was developed but not properly used, and that the end-to-end testing meant to validate the software against it doesn't appear to have been done.

That's the honest summary of all this. Specifying without verifying is a nice document. Verifying without specifying is guessing at what. You need both, in that order: first you decide what has to happen, then you check that's what happens.

The approved design had the math wrong. The ticket didn't say what to do if your seat got sold while you were looking at it. The server returned empty where everyone expected a number. The "archived" state didn't exist in the database. The number on screen wasn't the one deciding the delete. The test that was supposed to catch all of it couldn't fail. And a document from 1987 didn't clarify an ordering everyone found obvious.

None of that gets fixed by writing better code. It gets fixed earlier, by deciding, and later, by verifying.

Delegate the code. Don't delegate the judgment.