A crawler does not have to read your whole page to learn how it connects to the rest of your site. You can tell it in the HTTP response headers, before it parses a single line of HTML. That is what the Link header is for.
What it is
RFC 8288, “Web Linking,” defines the Link HTTP header: a way to express typed relationships between web resources in the response itself. It is the header-level equivalent of the <link> tags in your HTML <head>, but it arrives with the response, not buried in the body.
The syntax is a URL in angle brackets, followed by parameters:
Link: <https://example.com/page>; rel="canonical"The rel parameter names the relationship. You can list more than one link by separating them with commas.
Why AI engines care
Because the Link header comes back with the response, an agent can read it without rendering or parsing the page. Two relationships matter most here:
canonicaltells the agent which URL is the authoritative one, so it can collapse?utm=...and other variants down to a single source of truth, and not treat duplicates as separate pages.alternatewith a type can point at a machine-friendly representation, like the Markdown variant of the page.
For a crawler that does not run JavaScript, headers are a reliable, cheap channel for this kind of wayfinding.
How to set it up
You emit these from your server or CDN. A canonical pointer plus a Markdown alternate looks like this:
Link: <https://example.com/page>; rel="canonical",
<https://example.com/page.md>; rel="alternate"; type="text/markdown"In Nginx:
location /page {
add_header Link '<https://example.com/page>; rel="canonical", <https://example.com/page.md>; rel="alternate"; type="text/markdown"';
}In Express:
app.get("/page", (req, res) => {
res.set(
"Link",
'<https://example.com/page>; rel="canonical", <https://example.com/page.md>; rel="alternate"; type="text/markdown"',
);
// ...send the page
});Use registered relation names
Use registered relation names. canonical (defined in RFC 6596) and alternate are in the official IANA link relations registry, so conformant tools understand them. You will see rel="sitemap" in the wild, but it is not a registered relation and may simply be ignored. To point crawlers at your sitemap, the dependable route is still a Sitemap: line in your robots.txt, not a Link header. Reserve the Link header for canonical and alternate, where it carries real weight.