Google reorganized its guide on reducing crawl rate and added how to send Retry-After with 503 or 429 responses, Search Engine Roundtable reported on October 6.
What was updated
According to Search Engine Roundtable, Google reorganized its official documentation on reducing the crawl rate and added an explanation of the Retry-After header. The report is dated October 6, 2026.
The documentation says that when a server returns a 503 or 429 status code, it can also send the Retry-After header defined in RFC 9110. The value can be given either as a number of seconds to wait or as a UTC date and time.
Context and caveats
The report explains that this is not a new mechanism. Information about an already accepted method has been gathered into the crawl rate guide.
Per the official documentation, 500, 503 and 429 responses lower the crawl rate for the whole site (hostname), and the rate recovers automatically once errors decline. However, keeping this up for more than one or two days can hurt how the site appears in search results, so the guide advises using it only in emergencies.
It is aimed at situations such as an overloaded server, where you want to temporarily reduce visits from search crawlers. It is not something to configure in normal operation.
TAKEAWAY
What this means for your project
Most small sites will rarely need to do this themselves. Still, if a sale or announcement spikes traffic and slows your site, ask your developer or hosting admin whether crawling can be slowed temporarily in an emergency, and note that keeping it up beyond a day or two can hurt search visibility. Documentation can change, so check the current guide before acting.