Your HTML pages are built for people: navigation, sidebars, scripts, styling. An AI model has to wade through all of it to find the few hundred words that actually answer the question. A Markdown variant hands it the clean version directly.
What it is
A Markdown variant is a stripped-down, text-first copy of a page: the real content, in Markdown, with the nav and chrome removed. Sites that do this report large drops in token size, Cloudflare cites up to around 85% smaller, which means a model spends its limited context on your content instead of your layout.
This is a convention, not a formal standard. There is no RFC that says “serve Markdown to AI.” There are two established ways to do it, and one underlying piece (HTTP content negotiation) that is standard.
Why AI engines care
Most AI crawlers do not run JavaScript and do not want your styling. A clean Markdown representation is faster to fetch, cheaper to process, and less likely to be misread. It is the same instinct behind llms.txt: give the machine the signal without the noise. Real docs sites already do this, Anthropic, Stripe, Cloudflare, and Vercel among them.
How to set it up
Option A: a parallel .md URL. Serve a Markdown twin at a predictable path, usually the same URL with .md appended. Anthropic’s docs do exactly this: docs.claude.com/en/docs/intro.md returns clean Markdown instead of the rendered page. It is simple and easy to link to, including from your llms.txt.
Option B: content negotiation on the same URL. Keep one URL and return Markdown when the request asks for it via the Accept header. This is standard HTTP, the same mechanism browsers use to negotiate formats:
# Same URL, Markdown representation, selected by the Accept header.
curl https://yourdomain.com/your-page -H "Accept: text/markdown"Cloudflare offers this on its docs and blog: it detects Accept: text/markdown, converts the page, and returns Markdown. This option is generally the safer one for search, because you are not publishing the same content at a second crawlable URL.
Advertise it. Either way, tell agents the variant exists. A discovery hint in your HTML <head> works:
<link rel="alternate" type="text/markdown" title="Markdown version" href="/your-page.md" />You can do the same at the HTTP layer with a Link header.
Point .md URLs back to the original
If you go with separate .md URLs (Option A), point them back at the HTML original with a canonical signal. Search engines have long warned against serving materially different content at two separate crawlable URLs, and a canonical link keeps your HTML page as the authoritative one. Content negotiation (Option B) sidesteps that entirely, because there is only ever one URL.