# Help crawlers find current information

> Keep discovery files, dates and article provenance tied to real changes.

By Joey Kudish · Published 2026-10-31

Make it easy to discover new pages and distinguish current information from old information. These signals help describe your publishing, but do not prove a search engine has fetched a change or used it in an AI answer.

<h2 id="sitemap">XML sitemap</h2>

A sitemap lists the public URLs you want crawlers to discover. It should list working, preferred pages rather than redirects, missing pages or URLs excluded from indexing.

Generate it from published content and reference its real location in robots.txt. Remove stale URLs and check that each listed page is indexable and canonical. Use `lastmod` only when you know when the content changed. A sitemap is a discovery hint, not an indexing guarantee. Our [sitemap guide](/guides/sitemap) and Google's [sitemap overview](https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview) cover implementation.

<h2 id="date-modified">Freshness dates</h2>

An update date helps a reader judge whether a page is current. Disagreement between visible dates, structured data and sitemap dates makes that harder. A date far in the past may be perfectly accurate; the content decides whether it needs updating.

Correct impossible dates and align the signals to a genuine substantive update. Do not replace every date with today's date on each deploy. Check the served page and sitemap after publishing. Our [dateModified guide](/guides/date-modified) explains the fields; Google's [article guidance](https://developers.google.com/search/docs/appearance/structured-data/article) covers date markup for articles.

<h2 id="feeds">RSS and Atom feeds</h2>

RSS and Atom feeds provide a stream of new articles. They can help tools discover updates from a publishing site. A site with no ongoing publication may not need one.

If you publish regularly, generate a valid feed from live articles. Include stable item URLs and accurate dates. Advertise the feed with an alternate link in the page head. Open it in a feed reader and check a recent item. See [MDN's RSS overview](https://developer.mozilla.org/en-US/docs/Glossary/RSS). An empty feed added only to pass a check helps nobody.

<h2 id="authorship">Article authors</h2>

A byline and author profile let readers trace an article to a person or organization. The Scan checks article metadata and public author profiles. It does not judge whether an author is an expert.

Name the real author, provide a useful public profile and align visible bylines with JSON-LD. Correct generic placeholders or broken profile references. Do not invent qualifications. Check an article's HTML and the linked profile. Our [authority and proof guide](/guides/authority-and-proof) covers stronger evidence beyond a byline.

<h2 id="indexnow">IndexNow</h2>

IndexNow notifies participating search engines that a URL changed. It can reduce discovery delay, but a public Scan can only check conventional key-file locations. A plugin or CDN may keep the key elsewhere.

Check your CMS or CDN's existing integration before building one. Confirm recent submissions in Bing Webmaster Tools. If you need a custom integration, follow the [IndexNow documentation](https://www.indexnow.org/documentation), host the ownership key and send updates on real changes. Do not treat a missing conventional key as proof the integration is off. Our [IndexNow guide](/guides/indexnow) covers the common platforms.

<h2 id="reference-presence">Wikidata entry</h2>

A public reference such as Wikidata can help distinguish businesses with similar names. The Scan's match is a bounded reference lookup, not a judgment of importance or authority. A business without a reference has not failed a website requirement.

Check that any matched entity describes your business and links your official domain. Correct factual mistakes through the source's normal process. Do not create promotional Wikipedia pages or invent notability. Read [Wikipedia's conflict-of-interest guidance](https://en.wikipedia.org/wiki/Wikipedia:Conflict_of_interest) before editing. Improve the accurate information on your own site regardless of whether a reference exists.