Non-Determinism Doesn't Matter If the Tests Are Strict Enough
We verify behavior, not code.
“The big problem with AI is that an LLM generates something different every time.” I hear this a lot, mostly from developers. We’re used to determinism: you write something, it compiles or gets interpreted, and the result is the same every time. Here you write something and who knows what comes out. The complaint is fair, but I think it’s aimed at the wrong target. Unstable output isn’t the real problem. For decades we’ve been verifying something other than what we think we’re verifying.
My thesis: non-determinism in generation isn’t a problem if the spec and the tests are strict enough. “Strict” is the word doing the work here. I mean tests that pin down the required behavior, as opposed to tests that merely go green. With tests like that, the code can change from run to run as much as it likes. As long as it has to get through the wall of tests, the artifact at the output does exactly what’s required. So the thing worth investing in is the strictness of the specification and the tests, not a fight against non-determinism. (The obvious objection is that tests never cover everything. I’ll come back to it, because that’s the whole game.)
None of this is new. Our industry has always had the same problem. A business analyst writes requirements, and they pass through a developer’s head and interpretation into code that may well be flawed. The human translator from “where’s the spec?” to “works on my machine” was never deterministic: give two engineers the same requirements and you get two different solutions. That was never a problem, because we verify the behavior at the output, not the code: tests, QA, production metrics. There’s also code review, but review catches defects without proving correctness. It relies on human judgment about style, design, edge cases and maintainability. Whether the thing is correct is still settled by its behavior.
Where does the variance come from?
A fair counter-question to my own setup: why does the same prompt give different answers at all? Maybe because the spec is too shallow. A one-line ticket leaves a hundred decisions unmade, so we let the model swim, and it picks a different stroke each time. Tighten the spec, make it structured and unambiguous, and that freedom collapses. The model converges on the one thing the spec describes.
That’s largely true, and it’s the strongest version of the “non-determinism isn’t real” argument. Still, two things survive even a perfect spec.
The first is sampling. At any temperature above zero, the next token is drawn from a probability distribution. That comes from decoding, not from your spec, and no amount of prompt rigor removes it. Drop the temperature to zero (greedy decoding) and you’d expect determinism. That runs into the second problem.
The second is systems-level non-determinism. In practice even greedy decoding isn’t bit-for-bit reproducible, and the folklore explanation (floating point plus concurrency) is not the reason. As Thinking Machines Lab showed in 2025, the kernels lack batch invariance: operations like matmul and attention produce slightly different numbers depending on the total batch size, and in production the batch size shifts unpredictably with load. Your identical request shares a forward pass with whatever other requests happen to be in flight, and tiny floating-point differences propagate through the layers until one flips an argmax and you get a completely different output. This can be fixed. Thinking Machines Lab released batch-invariant kernels and demonstrated deterministic vLLM inference with a small upstream change. It isn’t the default, though. So “strict spec, identical output, every time” is false even mechanically, however good the prompt is.
The intuition that “only the variable names will differ” also undersells what’s left. Under a tight spec the model can still pick a different algorithm among equivalent ones, a different decomposition, error-handling style or library, and different behavior on edge cases the spec never mentioned. Variable names are the trivial end of that range.
Can’t we just specify everything? This is where it gets interesting. To remove every degree of freedom, the spec has to pin down every decision, and a spec that fixes every decision is no longer a spec. It’s isomorphic to the code. The asymptote of a “perfectly unambiguous spec” is the program itself. That gives a trade-off curve: as spec precision rises, semantic variance falls toward zero, but the cost of writing the spec rises toward the cost of writing the code, and the leverage you got from the model shrinks to nothing.
As the spec gets more precise, the model’s design-level variance falls, but the cost of authoring that spec climbs toward the cost of writing the code outright. Underneath sits a mechanical noise floor (sampling plus serving effects) that default inference never drives to zero.
So the problem splits in two. Semantic variance, where the model makes different design choices, depends on how complete the spec is. That’s the part that affects correctness, and you control it by tightening the spec until tightening costs more than it’s worth. Mechanical variance, sampling plus the systems-level noise above, usually has no bearing on correctness, so it mostly doesn’t matter: we verify behavior, not bytes. It matters only when byte-level reproducibility is itself the requirement (flaky evals, bug reproduction, audit trails, regulated pipelines), and then you reach for greedy decoding and batch-invariant kernels. Otherwise, tighten the spec to remove the variance that counts and let the rest go. The tests, the QA and the production metrics never cared whether the bytes were identical.
Hallucination inside the model also stops being scary once you picture a strict test suite as a wall. The agent generates code, possibly strange and unlike anything a human would write, and it hits the wall. Red: regenerate. Red again: regenerate again, until the tests go green. Whatever comes out the far side passed the wall by definition, so it does exactly what the tests demand. What the model did on the way there is none of our business, because only what clears the wall gets out.
Strict tests are a wall. Strange code bounces back for regeneration; only code that clears the wall becomes the artifact, which is therefore correct by construction.
This only works if the wall is solid. Weak, formal tests are a wall full of gaps, a sieve. Strange code slips through, and that’s when a hallucination becomes a defect. The fix is to leave no gaps in the wall, not to forbid the model from hallucinating.
Weak, formal tests are a sieve. Strange code passes unchecked, and the hallucination lands in the artifact as a defect.
Why does it feel like a catastrophe?
The “hmm, but are you sure it’ll write it correctly next time?” reaction has less to do with the model’s reliability than it seems. It’s a category error: a fear of mechanical variance dressed up as a question about correctness. We spent decades reading code as the artifact, so we learned to expect the same input to yield the same artifact, and now we instinctively read a difference in bytes as a defect. We never verified the artifact, though. We verified the behavior. So the requirement that supposedly scares us, confirming that the result is correct, isn’t new. We always charged the client for the value and also for someone separately confirming that the value was correct.
There’s a non-technical undercurrent too. AI is eating our lunch, often doing the job better, and somewhere inside we tensed up. The timing made it worse: the panic coincided with the correction of pandemic-era over-hiring, not with an AI breakthrough. AI made a convenient banner to wave at investors, a move now called “AI washing.” That’s the backdrop, though, not the cause. The cause is that we substitute the artifact for the behavior, some of us without noticing and some of us deliberately.
Verification is being delegated too
Since we verify behavior and not bytes, verification itself is being delegated to agents too, and here again they’re often better than us. It’s already happening. In mid-2026 an autonomous detection harness, Code-Augur, found 22 previously unknown vulnerabilities in widely used open-source projects. These holes had sat in human-written code for years, in components treated as the gold standard of engineering, and nobody saw them until that same “dumb” agent walked the code and surfaced them.
The trap is symmetric, though. A spec of what to verify is still a spec, and “just make it work” is as vague a ticket as “write me a function.” Push all the rigor onto the verification side and it doesn’t go away; you’ve only relocated it. If your acceptance criterion is “well, it seems to run,” you haven’t verified anything. You’ve moved the same under-specification from the code onto the test. The behavior has to be nailed down as fussily as we’d want the implementation nailed down, otherwise verification gets delegated as sloppily as code once got written. In any other field the cost of that step is taken for granted. Imagine ordering wallpaper and finding two line items in the estimate: “hanging: 100” and “checking it was hung correctly, otherwise re-hanging: 150.” Yet in IT nobody bats an eye.
So the work doesn’t disappear; it moves up a level. Writing code stops being the substance of the job. The substance becomes saying precisely what should happen and checking rigorously that it did. That isn’t less work, just different work.
What comes after imitation
Agents emulate us, they do it very well, and they’ll keep getting better at it. What I find interesting is when the work will stop imitating humans and move to a level we can’t reach, limited as we are by a physiology a machine doesn’t share. And what kind of code will agents write once no human needs to read it at all? The whole layer of review we built around readability (style, consistency, maintainability) exists only because a human had to read the artifact next. Take that reader away and only the verification of behavior survives. What will code look like when none of it was written to be read?