llms.txt is a proposed web standard — a markdown file placed at a site's root that gives 📝Large Language Model (LLM)s a curated, plain-text map of a website's most important content for use at inference time.
The format was proposed in September 2024 by Jeremy Howard, co-founder of the AI research lab Answer.AI. It addresses a structural limitation in how language models read the web: context windows are too small to ingest entire websites, and converting HTML pages — thick with navigation, ads, and JavaScript — into clean text is imprecise and wasteful. An llms.txt file resolves this by publishing a concise, human- and machine-readable summary with curated links, so a model can locate and load only the content that matters.
Functionally, llms.txt is analogous to robots.txt and the XML sitemap, but where those govern how search crawlers access a site, llms.txt curates content for language-model readability. The specification, hosted at llmstxt.org, defines a fixed markdown structure: an H1 project name, a blockquote summary, and sections of annotated links, with an optional companion /llms-full.txt that inlines a site's full documentation in one file. Early adoption by 📝Anthropic, Mintlify, and other documentation platforms has positioned it as an emerging convention within 📝Generative Engine Optimization (GEO).
Inside 📝MythOS we generate a dynamic llms.txt automatically from every 📝Public Memo a 📝Creator publishes — so a library extends itself outward, optimized for ingestion by the models that increasingly mediate discovery.