System design template

Search autocomplete architecture.

This is a latency problem wearing a search problem's clothes: past about a hundred milliseconds users have already typed the next character and your response is worthless.

Download for macOS
v0.1.33 · .dmg · Apple Silicon & Intel
Signed & notarized by Apple · opens without a Gatekeeper warning
sha256 698955a0187bc039f4c74f5d05a9f10fbb27376a45788a0a241d1326b73873c7
Download for Windows instead
$curl -fsSL https://lucidtrain.com/install.sh | sh
Search autocomplete architecture. 11 components across 4 tiers.
Search autocomplete architecture. Rendered by the same ELK layout engine the app runs: the agent emits components, tiers and edges, and the engine places them, so the boxes cannot overlap.

The components

Every row below is read from the graph that produced the diagram above, so the two cannot disagree.

ComponentTierWhy it is there
Search boxClientDebounced, cancels in-flight requests
Edge cacheEdgePrefixes are extremely cacheable
API gatewayEdgeSupporting component
Suggest serviceApplicationp99 under 100ms or the feature is useless
Trie serviceApplicationTop-k precomputed at each node
PersonalisationApplicationSupporting component
Index builderApplicationRebuilds hourly, swaps atomically
Query aggregatorApplicationSupporting component
RedisDataSupporting component
Query logDataSupporting component
WarehouseDataSupporting component

Design decisions worth arguing about

A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.

Precompute the top results at every prefix

Ranking candidates at request time cannot meet the latency budget, so the ranking is done in advance and stored at each trie node. Lookup becomes a walk to a node and a read. The cost is that suggestions are as fresh as the last build, which is why breaking news is the standard example of autocomplete looking stupid, and why the rebuild interval is a product decision rather than an infrastructure one.

Prefixes are extremely cacheable

Query distributions are heavily skewed, so a small cache covers a large share of traffic, and prefixes are short and identical across users. Caching at the edge removes most requests from your infrastructure entirely. Personalisation breaks this, because a personalised response is unique per user, which is the real argument for keeping personalisation as a light reordering of a cacheable base result.

Rebuild and swap, never mutate in place

Updating a live trie under query load means locking or accepting inconsistent reads. Building a new one and swapping the pointer makes updates atomic and instant, and lets you validate the new index before it serves anything. It costs double the memory during the swap, which is the price of never serving a half-updated index.

Debounce on the client, because most keystrokes are noise

Firing a request per character means most requests are for prefixes the user is typing through and will never see results for. A short debounce and cancelling in-flight requests removes the majority of load before it exists. It adds perceived latency equal to the debounce, so this is tuned in tens of milliseconds and is worth measuring rather than guessing.

How it changes with scale

Request volume is roughly typing speed times users, so it is very high but also very cacheable and very uniform. The trie's memory grows with vocabulary rather than traffic, which means the index is often small enough to sit in memory on every node, removing a network hop from the critical path entirely.

Where it breaks first

A stale or failed index build. Because the serving path keeps working with the previous index, nothing errors and latency is unaffected; suggestions simply stop reflecting reality. The only way this gets noticed is by monitoring build freshness explicitly, which is the kind of alert that exists only after the first time it happens.

Draw this yourself

Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.

shell
$ Diagram search autocomplete: a debounced client, edge caching of prefixes, a suggest service backed by a trie with precomputed top-k at each node, personalisation, query log aggregation into a warehouse, and an hourly index builder that swaps atomically.

When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.

FAQ

Questions about this design

Trie or a search engine?

A trie when the requirement is genuinely prefix completion, because it is faster and simpler. A search engine when you need fuzzy matching, typo tolerance and relevance signals, which most product searches eventually do.

How do you handle typos?

Edit-distance expansion at query time, or by indexing common misspellings from the query log. The log-based approach is usually better because it captures the typos your users actually make rather than every possible one.

Should suggestions be personalised?

Lightly. Recent personal searches ranked above global ones covers most of the benefit while keeping the base result cacheable. Fully personalised suggestions destroy your cache hit rate for a small quality gain.

How do you keep offensive suggestions out?

A blocklist applied at build time rather than at query time, so the filtering cost is paid once per rebuild rather than on every request. It also means the filter is auditable, since you can inspect the index.

More templates