Andrej Karpathy's three-paradigm frame is the cleanest map of what just happened to programming. Software 1.0 is source code compiled to binaries. Software 2.0 is curated datasets compiled to neural-network weights by an optimizer. Software 3.0 is natural-language prompts executed by a large language model — programs written in English, with measurably different performance depending on how they're written. This leaf is the frame, the verified numbers behind it, and the honest read on what it doesn't mean.
+--------------+------------------------+----------------------------------+ | Paradigm | Source material | "Compiler" | +--------------+------------------------+----------------------------------+ | Software 1.0 | Explicit source code | Compiler/interpreter -> binary | | Software 2.0 | Curated datasets | Optimizer (SGD/Adam) -> weights | | Software 3.0 | Natural-language prompt| LLM inference -> behaviour | +--------------+------------------------+----------------------------------+
The point of the frame is not that each paradigm replaces the last — they stack. The Linux kernel is still 1.0. A driving perception stack is 2.0: nobody writes the lane-detection algorithm; a team curates the dataset that the optimizer compiles into weights. And an agent following a system prompt is 3.0: the behaviour was specified in English, and changing the English changes the program. Karpathy's one-liner — the hottest new programming language is English — is a claim about where specification moved, not about the death of code.
Distilled from two Karpathy talks — the "Software 3.0" keynote and his Stanford CS25 transformer lecture — with every load-bearing claim chained to its paper. Two corrections applied to the source synthesis: Karpathy's famous ImageNet human-baseline exercise covered the challenge's 1,000 categories (not 10,000), and the in-context-learning-as-gradient-descent result in §05 is stated as evidence in simplified settings, not the "proof" it is often inflated into.
Software 2.0's development loop — built at industrial scale in Karpathy's Tesla years — is the data engine: ship the model, harvest the failures, label the edge cases, retrain, repeat. The engineers' daily work is not authoring algorithms but curating the dataset that compiles into them. The 2017 "Software 2.0" essay called this shift before it was consensus: version control for datasets, test suites as labelled evaluation sets, and debugging as failure-mode triage.
Why it earned the "software" name: weights are a binary produced from source material by a deterministic build step. What changed is which artefact humans edit. That same move — asking "what do humans edit now?" — is exactly how to understand 3.0.
The strongest argument that a prompt is a program, not a wish, is that its phrasing moves benchmark numbers the way an algorithm change would. The canonical result, verified at source:
Kojima et al. (2022) added five words — "Let's think step by step" — before the answer, and accuracy on the MultiArith arithmetic benchmark went from 17.7% to 78.7%; GSM8K went from 10.4% to 40.7% (text-davinci-002, so these are 2022-era numbers — the era matters). Zhou et al.'s Automatic Prompt Engineer then searched for a better program and found one: "Let's work this out in a step by step way to be sure we have the right answer" — 82.0%. A machine optimising the wording of an English program for measurable performance: that is Software 3.0 in one experiment.
The mechanism is not mystical. A model spends a roughly fixed amount of compute per token, so a prompt that elicits intermediate reasoning tokens buys the computation more serial steps — a scratchpad. That thread runs forward directly into test-time compute: what began as five magic words in 2022 became reasoning models trained to do it natively, with verification as the binding constraint.
Two months after ChatGPT launched a thousand demos, Jonas Degrave's stood out because it made the paradigm concrete: he prompted the model into being a stateful Linux terminal — fake filesystem, command execution, believable network round-trips — no VM anywhere, the whole machine simulated in activations. The other early tell was Bing's "Sydney": when its system prompt leaked, it showed that an entire product persona — behavioural bounds, safety policy, tone — was instantiated as a page of English.
Together they define the 3.0 runtime: a general-purpose engine whose behaviour is configured, not compiled — which is why prompt files, system prompts and agent instructions deserve the treatment source code gets: versioned, reviewed, tested. That discipline has its own leaf: spec-driven development is what Software 3.0 grows into when it stops being vibes.
The 3.0 runtime exists because the field converged on one architecture. Before 2012, vision, speech and NLP each had bespoke, handcrafted features; AlexNet showed scale beats handcraft, and the 2017 transformer unified the architectures — vision (ViT), speech (Whisper), RL (Decision Transformer), structural biology (AlphaFold) all run attention-centric backbones. Karpathy's CS25 telling of the lineage is worth knowing: Bahdanau's soft attention (2014) solved the Seq2Seq bottleneck, and — by Bahdanau's own account — it was Bengio who suggested the name "attention" in revision. Transformers won on three axes at once: expressive (attention over sets), optimisable (residuals + LayerNorm), and shaped like the hardware (shallow-and-wide parallel graphs on GPUs).
Two grounding facts for the working engineer. First, the whole thing is readable: nanoGPT reproduces GPT-2 (124M) with a ~300-line model definition plus a ~300-line training loop — OpenWebText, one 8×A100 node, about four days, validation loss ~2.85. The mystique of the substrate does not survive a weekend with that repo. Second, the honest version of a widely inflated claim: research (von Oswald et al., 2022) shows that linear attention can implement gradient-descent-like updates on toy regression tasks in its forward pass — suggestive evidence that in-context learning is implicit optimisation, not a proof that production LLMs learn this way. The question is open; the leaf says so.
Software 3.0 lowers the capital cost of capability: a small SA team with strong specification discipline now ships systems that used to need a frontend team, a data team and a research budget. But the frame cuts both ways — if the program is English, then vague English is a buggy program, and the differentiator shifts from who can code to who can specify and verify. That's the skills ladder worth building here: treat prompts and agent instructions as reviewed artefacts in the repo, date and re-test the behaviour claims your systems rest on, and let the 1.0/2.0/3.0 question — what are we actually editing? — drive where scarce engineering time goes.
Every number in this leaf chains to one of these. Last reviewed 2026-08-23.