Neural networks are not just another classifier. They are the beginning of a fundamental shift in how software gets written.
📝Andrej Karpathy coined Software 2.0 in a November 11, 2017 Medium essay to name the moment when the dominant artifact of programming stopped being human-authored source code and became a set of learned weights. In the new paradigm, the programmer's job is to curate a dataset and choose an architecture; the network compiles itself by training. The teams split — 1.0 programmers maintain the surrounding infrastructure, and 2.0 programmers grow and clean the data that becomes the program.
The convergence around the framing has only sharpened with time. Pete Warden wrote in parallel about deep learning as "self-writing programs." The broader machine learning community absorbed the language quickly, with practitioners describing model weights as the new binaries and Hugging Face emerging as the GitHub of the 2.0 stack. By 2025, Karpathy had extended the lineage publicly with Software 3.0 — the era in which natural language prompts to LLMs become the programming interface itself, with the "hottest new programming language" being English.
The application is to notice which layer of the stack is currently absorbing the most leverage. In 1.0 the leverage was in better languages and editors. In 2.0 it moved to data pipelines and training systems. In 3.0 it moves again — to context, knowledge bases, and the orchestration of agents (see 📝Never Felt So Behind and 📝Karpathy's Flag in the Ground — LLM Knowledge Bases). The pattern beneath all three transitions is the same: the work moves up the stack faster than the discourse, and the people who notice early build the infrastructure the rest of the field eventually needs.