Stanford CS329A: Self-Improving AI Agents is a graduate course at 📝Stanford University on techniques that let AI agents improve themselves through interaction with their environment and with their own outputs.
Co-taught by Aakanksha Chowdhery and Azalia Mirhoseini in Autumn 2025, the course was released publicly by Stanford Online as a 📝YouTube lecture series in August 2026. Its syllabus spans test-time compute scaling, verification, learning from execution feedback, training-time reinforcement learning, search-augmented reasoning, memory, evolutionary design of agentic systems, long-horizon evaluation, and multimodal robotics, with guest lectures from researchers at 📝Google 📝DeepMind and Physical Intelligence.
The first lecture, a 70-minute course overview, traces how scaling compute, data, and parameters drove progress through 📝GPT-3 and PaLM before showing signs of saturation, and how post-training — instruction tuning and RLHF — produced the jump to 📝ChatGPT. It then frames inference as the new frontier: repeated sampling paired with verifiers lets smaller models outperform larger ones, and the synthetic data generated at test time can feed back into training, closing a self-improvement loop.
The lecture names the generator–verifier gap as the central bottleneck: producing candidate answers is cheap, but judging their usefulness requires feedback, which is hardest to obtain in non-verifiable domains such as creative writing. It defines an agent as a system that pursues a goal by planning, acting in an environment, correcting from feedback, and deciding when to stop, and observes that most deployed systems remain agentic workflows — prompt chains, routers, parallelization, and orchestrator patterns — rather than fully open-ended self-improving loops.