Skip to content
Glossary · definition

Crawl budget

The finite attention a search engine spends on a site. It becomes a live problem when crawlable URLs can be generated without limit — faceted navigation being the usual cause — because the budget is then spent on pages nobody searches for.

Also written Crawl efficiency. Atlas writes British English; the definition is the same.

01

Why it matters

The consequence, not the concept

It is almost always an architecture problem wearing a technical costume. Capping crawl of generated URLs is the fix; asking for more crawling is not, because the budget was never the constraint — the URL space was.

02

Where Atlas works on this

2 routes
04

Build visibility

Seven questions

A definition is only worth publishing if it is true of your site too. Atlas reads the site before the first conversation.

Crawl budget Build visibility