For years, making models better meant scaling two things: parameters and data. Test-time compute is the third axis — spending more inference to get better answers from the same weights, by sampling many candidates, searching, and checking. The headline result is startling: a small model sampled thousands of times can out-solve a frontier model's single attempt. The catch is the whole story: those gains only convert into real performance where an automatic verifier exists. Reasoning models are this idea productised; the generator-verifier gap is its bottleneck; and the practical question for your stack is when sampling-plus-checking beats simply buying the bigger model.
The first two scaling axes are settled science: the 2020 scaling-laws work (Kaplan et al., OpenAI) showed loss falls predictably as you grow compute, parameters, and data, and Chinchilla (Hoffmann et al., DeepMind, 2022) refined the recipe — scale tokens with parameters. That era took models from BERT's ~340M parameters to the hundreds of billions, and unlocked in-context learning and, past roughly the 100B mark, the chain-of-thought reasoning gains Wei et al. documented in 2022.
Test-time compute is the axis that doesn't touch the weights. Instead of training a better model, you spend more inference on the problem: sample many candidate answers in parallel, let the model think longer before answering, search over intermediate steps, and use a checker to pick the winner. Pre-training buys capability once; test-time compute rents extra performance per query — which is exactly why its economics, not just its accuracy curves, decide when it's the right tool.
The cleanest evidence is Large Language Monkeys: Scaling Inference Compute with Repeated Sampling (Brown et al., 2024 — Stanford's Scaling Intelligence Lab). Sample a model repeatedly on the same problem and measure coverage — the fraction of problems where any sample is correct. Coverage keeps climbing across four orders of magnitude of samples, often log-linearly — an inference-time scaling law to sit beside the training-time ones. In their experiments, a small open model like Llama-3 8B, sampled up to 10,000 times per problem with an automatic checker, reaches coverage on hard math and coding benchmarks that exceeds a frontier model's single-attempt (pass@1) score.
Two practical notes travel with the result. Parallel sampling trades compute, not latency — the candidates generate concurrently, so wall-clock stays flat while the bill grows. And diversity has a ceiling: push sampling temperature much past ~1.2 and generations degrade into incoherence (a practitioner's bound worth knowing, from the CS329A lecture treatment of the paper).
This leaf draws partly on Stanford's CS329A (Self-Improving AI Agents) course treatment of these results — and its co-instructor, Azalia Mirhoseini, is an author of the Monkeys paper itself. The lecture is a primary author explaining her own work: authoritative on what the paper says, not an independent replication of it. The paper and its public code are the checkable sources.
Here is the part the headline hides. Coverage counts a problem "solved" if any of your 10,000 samples is right — but in production you have to pick one answer. The Monkeys authors are explicit: the gains convert into real performance in domains where answers can be automatically verified — code with unit tests, formal proofs, math with checkable answers. Where no such verifier exists, identifying the correct sample among thousands becomes the problem, and common tricks like majority voting or reward-model scoring plateau well below the coverage ceiling.
That is the generator-verifier gap: for many tasks, checking an answer is as hard as — or harder than — producing one, and verification, not generation, becomes the bottleneck. The research response is to build better checkers: Let's Verify Step by Step (Lightman et al., OpenAI, 2023) showed a process reward model — grading each reasoning step — substantially outperforms an outcome-only grader on hard math. The engineering response is the one this tree keeps arriving at from every direction: the same law that governs specs governs sampling. A spec without verification is a vibe with headers; ten thousand samples without a verifier are ten thousand vibes. Test-time compute is only as good as the check you can run against it.
Chain-of-thought started as a prompting trick; modern reasoning models internalise it. OpenAI's o1 announcement showed accuracy on hard competition math (AIME) rising log-linearly with test-time compute — the same curve shape as the sampling law, now inside one model's "thinking." Instead of you orchestrating samples and verifiers externally, the model performs search, decomposition, hypothesis testing, self-evaluation, and backtracking natively before answering — trained into the weights with reinforcement learning, spent at inference as thinking tokens.
The consequence is on your invoice. A reasoning model converts the sampling bill into a thinking-token bill — the Kimi K3 leaf's ~13,000-token answer to a simple drawing prompt is what always-on test-time compute looks like as a line item. Test-time scaling didn't make intelligence cheaper; it made extra intelligence purchasable per query — which is precisely why routing easy work away from it matters.
Three ways to buy accuracy at inference. The verifier decides which one is rational — and for a rand-denominated budget, the difference is large.
For a team paying for inference in dollars, sample-and-verify is the rare strategy that gets better as your model gets cheaper: a budget open-weight model plus a real verifier plus serving you control is a rand-friendly substitute for frontier calls — on verifiable work. It composes with the router argument from the ownership series: route easy work to cheap single-shot, verifiable hard work to cheap sample-and-verify, and reserve the premium reasoning model for the judgment tail no verifier covers. The investment that unlocks all of it is the same one the spec leaf argues for: build the checks.
The papers first, the course as the guided tour. GSM8K is Cobbe et al. (OpenAI, 2021) — often mis-attributed to the MATH paper. Last reviewed 2026-08-05.