The web is developing a new access layer — one that sits alongside traditional HTTP pages and search indexing, but is designed specifically for AI agents. A public 📝Model Context Protocol (MCP) server is the mechanism by which any website can participate in that layer. Sites that publish one become legible to the agentic web; sites that don't risk becoming invisible to it.
This memo is a reference for website owners and developers considering whether and how to build a public MCP.
What a public MCP server is
MCP is an open protocol, developed by Anthropic and now maintained as an open standard, that defines how AI agents interact with external data sources and tools. An MCP server exposes a website's content or functionality as a set of named, typed, documented "tools" that any compatible agent can invoke.
A public MCP server specifically is one that requires no authentication and is openly accessible to any agent or crawler. It is read-only by design. Its purpose is exposure — making a site's content available for agents to query, summarize, cite, and act on.
The analogy to robots.txt and sitemaps is useful but incomplete. Those files tell crawlers what exists and what they're allowed to access. A public MCP server tells agents what a site contains and gives them a structured interface to query it intelligently — with typed inputs, filtered results, and responses shaped for machine consumption.
Why this matters now
Search engine optimization was built around a model where humans type queries into a box and choose from a ranked list of links. That model is eroding. A growing share of information retrieval is being mediated by AI agents — systems that don't click through to pages, don't browse, and don't rank results. They call tools, receive structured data, and synthesize responses.
In this environment, content that is not queryable by agents is content that doesn't exist for them. A well-written blog post buried in HTML, with no structured access layer, is harder for an agent to use than a sparsely written document available through a clean MCP tool. Structure and accessibility matter more than prose quality when the consumer is a machine.
This shift is not speculative. Claude, GPT-based agents, and an expanding ecosystem of third-party AI products already support MCP. The infrastructure for agents to discover and connect to MCP servers — registries, auto-discovery conventions, client integrations — is consolidating quickly.
SEO and GEO impact
Search Engine Optimization (SEO) and Generative Engine Optimization (GEO) are converging concerns for any site that wants to be found.
Traditional SEO focuses on ranking in search result pages. GEO is the discipline of structuring content so that AI models cite it, surface it, and treat it as a primary source in generated responses. A public MCP server is a GEO accelerant because it removes friction from the citation pathway: instead of an agent having to crawl, parse, and interpret HTML, it can call a tool and receive clean, structured, citable content directly.
Canonical URLs in responses are the most direct SEO mechanism. Every tool response from a well-built public MCP should include the canonical URL of the originating page. When an agent uses that content in a response that gets published anywhere on the web — a blog post, a forum answer, an AI-generated summary — the canonical URL travels with it. This is a new class of inbound link generation that operates outside traditional content marketing.
Google AI Overviews and equivalent surfaces increasingly prefer structured, machine-readable sources. A public MCP returning consistently formatted content with authorship metadata, timestamps, and canonical URLs provides exactly the signals these systems favor.
Tool description quality is an underappreciated ranking signal. When an agent's orchestration layer decides which MCP to call for a given query, it reads tool descriptions the same way a search engine reads meta descriptions. A tool described as "search articles" competes poorly against one that specifies the content domain, query types it handles well, and example use cases. Tool descriptions are effectively SEO copy for the agentic web.
Discoverability mechanisms
A public MCP server that no agent can find is just an API. Discoverability is a distinct problem from implementation, and it has several active solutions:
/.well-known/mcp.json is the emerging auto-discovery convention. Placing this file at the root of a domain allows MCP-aware agents to find and configure a site's MCP endpoint automatically, without manual setup or directory submission. It is the agent-web equivalent of a DNS record — infrastructure-level discoverability.
MCP registries (mcp.so, Glama, Smithery, and others emerging) are curated directories of MCP servers. Being listed increases the probability that agents select a server when handling queries relevant to its content domain. Most are free for open public servers.
llms.txt is a voluntary standard (analogous to robots.txt) that allows a site to declare its content structure, MCP endpoint, and guidance for how language models should weight and navigate its content. It is increasingly respected as a signal by model providers doing web crawls.
Streamable HTTP transport is the current MCP spec standard for remote public servers, preferred over the earlier SSE transport for new deployments. Using it ensures compatibility with current and future MCP clients.
What to expose
For most content-focused websites, three tools cover the primary agent use cases:
- A search tool — full-text or semantic search across public content, filterable by category, date, or tag. Returns titles, excerpts, canonical URLs, and metadata.
- A content retrieval tool — fetch a single piece of content by ID or slug. Returns full body, metadata, and canonical URL.
- A taxonomy/navigation tool — enumerate the site's categories, tags, or topics with item counts. Allows agents to understand what the site covers before querying.
Beyond these, tool design should follow the site's natural content model. A recipe site might expose cuisine and cook-time filters. A documentation site might expose version-scoped search. The tools should reflect how a knowledgeable agent would want to navigate the content, not just mirror the database schema.
What to withhold
A public MCP server is explicitly read-only and explicitly public. It should expose nothing that is not already publicly accessible on the website:
- No content behind authentication or paywalls
- No user-specific or personalized data
- No write operations of any kind
- No internal system information
Rate limiting is non-negotiable. An open, unauthenticated endpoint will receive automated traffic at volume. IP-based rate limiting at the edge (via Cloudflare or equivalent) is the standard approach.
The case for acting early
The MCP registry and discoverability ecosystem is consolidating now. Early entrants in any new web standard benefit disproportionately from the period when competition is low and discovery surfaces are not yet saturated. The cost of building a minimal public MCP server — three tools, a single deployment, two discoverability files — is low relative to the surface area it opens.
For sites with substantial, structured, high-quality content, a public MCP is also a defensible position: the content corpus is the product, and making it queryable by agents without losing canonical attribution means participating in the agentic web on your own terms rather than being extracted from without credit.
The framing that resonates most: a public MCP server is not optional infrastructure for the future web — it is table stakes for any site that wants to remain findable. The transition from human-browsed to agent-mediated information retrieval is not arriving; it is underway. Sites that build this layer early will compound the benefits. Sites that wait will face a discoverability gap that looks like the gap that opened between early and late adopters of structured data markup in the 2010s — visible in hindsight, invisible until it hurts.