System design template

Distributed web crawler architecture.

The engineering problem is not fetching pages quickly. It is fetching them politely, at scale, without visiting the same content a thousand times.

Download for macOS
v0.1.33 · .dmg · Apple Silicon & Intel
Signed & notarized by Apple · opens without a Gatekeeper warning
sha256 698955a0187bc039f4c74f5d05a9f10fbb27376a45788a0a241d1326b73873c7
Download for Windows instead
$curl -fsSL https://lucidtrain.com/install.sh | sh
Distributed web crawler architecture. 12 components across 4 tiers.
Distributed web crawler architecture. Rendered by the same ELK layout engine the app runs: the agent emits components, tiers and edges, and the engine places them, so the boxes cannot overlap.

The components

Every row below is read from the graph that produced the diagram above, so the two cannot disagree.

ComponentTierWhy it is there
Seed listClientSupporting component
URL frontierApplicationPriority plus politeness delay per host
Fetcher poolApplicationRespects robots.txt and crawl delay
Robots cacheApplicationSupporting component
ParserApplicationSupporting component
DedupApplicationContent hash, not URL: mirrors are everywhere
IndexerApplicationSupporting component
KafkaDataSupporting component
Seen setDataBloom filter, false positives are acceptable
Raw pagesDataSupporting component
Search indexDataSupporting component
The webExternalSupporting component

Design decisions worth arguing about

A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.

Politeness is a per-host constraint on a global queue

A naive priority queue will happily send a thousand simultaneous requests to one host, which is indistinguishable from an attack. The frontier therefore has to be structured per host, with its own delay, while still supporting global prioritisation. That dual requirement is the reason the frontier is a component rather than a data structure, and it means the crawler is often idle on high-priority work because the relevant host is in a cooldown.

Deduplicate on content, not on URL

The same page is reachable through many URLs: tracking parameters, session ids, mirrors, print views. URL-based deduplication catches almost none of it. Hashing normalised content catches all of it, at the cost of fetching the page before you can tell, so you pay the bandwidth either way and save only the parsing and indexing.

A Bloom filter, accepting that it lies occasionally

Tracking every URL ever seen exactly requires memory proportional to the web. A Bloom filter answers in constant space with false positives but never false negatives, meaning the crawler occasionally skips a page it has not actually seen. For a crawler that is an acceptable loss, and choosing it is a deliberate decision to trade completeness for tractability rather than an optimisation.

Store raw pages before parsing them

Parsers change constantly, and reparsing from stored HTML is far cheaper and far politer than recrawling. Keeping raw content makes the extraction logic iterable. It costs a great deal of storage, which is the main reason people skip it and then discover the cost the first time they need to change an extractor.

How it changes with scale

Throughput is bounded by politeness rather than bandwidth: you can only go so fast per host, so crawling faster means crawling more hosts concurrently. That makes frontier partitioning by host the natural sharding key, and it makes the long tail of small sites, rather than the large ones, the thing that determines total coverage.

Where it breaks first

A crawler trap. Dynamically generated pages that link to more dynamically generated pages, calendars being the classic case, produce an infinite supply of unique URLs with no new content. Without depth limits, per-host budgets and content-based deduplication, the crawler spends itself entirely on one site and never notices.

Draw this yourself

Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.

shell
$ Diagram a distributed web crawler: seed list into a URL frontier with per-host politeness, a fetcher pool respecting robots.txt, raw page storage, a parser extracting links, content-hash deduplication with a Bloom filter, and an indexer.

When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.

FAQ

Questions about this design

How strictly should robots.txt be followed?

Strictly, and cache it with a sensible TTL rather than fetching it per request. Ignoring it gets you blocked, and at any scale it gets you blocked by the hosts you most wanted.

How do you decide what to crawl next?

Some combination of expected value and freshness: pages that change often and matter often, recrawled sooner. This is where crawlers actually differ, and it is much more consequential than fetch speed.

Should you render JavaScript?

It costs one to two orders of magnitude more per page than fetching HTML, so it is a per-site decision rather than a default. Many sites still expose the content in the initial response or a feed if you look.

How do you avoid recrawling unchanged pages?

Conditional requests with ETag and If-Modified-Since. A 304 costs a round trip and no bandwidth, and well-behaved sites support it, which makes it the cheapest freshness mechanism available.

More templates