Search autocomplete architecture.
This is a latency problem wearing a search problem's clothes: past about a hundred milliseconds users have already typed the next character and your response is worthless.
The components
Every row below is read from the graph that produced the diagram above, so the two cannot disagree.
| Component | Tier | Why it is there |
|---|---|---|
| Search box | Client | Debounced, cancels in-flight requests |
| Edge cache | Edge | Prefixes are extremely cacheable |
| API gateway | Edge | Supporting component |
| Suggest service | Application | p99 under 100ms or the feature is useless |
| Trie service | Application | Top-k precomputed at each node |
| Personalisation | Application | Supporting component |
| Index builder | Application | Rebuilds hourly, swaps atomically |
| Query aggregator | Application | Supporting component |
| Redis | Data | Supporting component |
| Query log | Data | Supporting component |
| Warehouse | Data | Supporting component |
Design decisions worth arguing about
A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.
Precompute the top results at every prefix
Ranking candidates at request time cannot meet the latency budget, so the ranking is done in advance and stored at each trie node. Lookup becomes a walk to a node and a read. The cost is that suggestions are as fresh as the last build, which is why breaking news is the standard example of autocomplete looking stupid, and why the rebuild interval is a product decision rather than an infrastructure one.
Prefixes are extremely cacheable
Query distributions are heavily skewed, so a small cache covers a large share of traffic, and prefixes are short and identical across users. Caching at the edge removes most requests from your infrastructure entirely. Personalisation breaks this, because a personalised response is unique per user, which is the real argument for keeping personalisation as a light reordering of a cacheable base result.
Rebuild and swap, never mutate in place
Updating a live trie under query load means locking or accepting inconsistent reads. Building a new one and swapping the pointer makes updates atomic and instant, and lets you validate the new index before it serves anything. It costs double the memory during the swap, which is the price of never serving a half-updated index.
Debounce on the client, because most keystrokes are noise
Firing a request per character means most requests are for prefixes the user is typing through and will never see results for. A short debounce and cancelling in-flight requests removes the majority of load before it exists. It adds perceived latency equal to the debounce, so this is tuned in tens of milliseconds and is worth measuring rather than guessing.
How it changes with scale
Request volume is roughly typing speed times users, so it is very high but also very cacheable and very uniform. The trie's memory grows with vocabulary rather than traffic, which means the index is often small enough to sit in memory on every node, removing a network hop from the critical path entirely.
Where it breaks first
A stale or failed index build. Because the serving path keeps working with the previous index, nothing errors and latency is unaffected; suggestions simply stop reflecting reality. The only way this gets noticed is by monitoring build freshness explicitly, which is the kind of alert that exists only after the first time it happens.
Draw this yourself
Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.
When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.
Questions about this design
More templates
- URL Shortener System DesignThe canonical read-heavy system: roughly a hundred reads for every write, and a redirect that has to be fast enough that nobody notices it happened.
- Chat Application System DesignLong-lived connections change everything: the hard part is not storing messages, it is knowing which of your servers is holding the socket you need to write to.
- Payment System DesignThe only system on this list where being approximately right is indistinguishable from being wrong, and where the provider, not you, holds the truth.
- RAG Pipeline ArchitectureAlmost every RAG system that disappoints is failing at retrieval, not generation, and the architecture is what decides whether you can tell.