Search autocomplete architecture.
This is a latency problem wearing a search problem's clothes: past about a hundred milliseconds users have already typed the next character and your response is worthless.
The components
Every row below is read from the graph that produced the diagram above, so the two cannot disagree.
| Component | Tier | Why it is there |
|---|---|---|
| Search box | Client | Debounced, cancels in-flight requests |
| Edge cache | Edge | Prefixes are extremely cacheable |
| API gateway | Edge | Supporting component |
| Suggest service | Application | p99 under 100ms or the feature is useless |
| Trie service | Application | Top-k precomputed at each node |
| Personalisation | Application | Supporting component |
| Index builder | Application | Rebuilds hourly, swaps atomically |
| Query aggregator | Application | Supporting component |
| Redis | Data | Supporting component |
| Query log | Data | Supporting component |
| Warehouse | Data | Supporting component |
Design decisions worth arguing about
A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.
Precompute the top results at every prefix
Ranking candidates at request time cannot meet the latency budget, so the ranking is done in advance and stored at each trie node. Lookup becomes a walk to a node and a read. The cost is that suggestions are as fresh as the last build, which is why breaking news is the standard example of autocomplete looking stupid, and why the rebuild interval is a product decision rather than an infrastructure one.
Prefixes are extremely cacheable
Query distributions are heavily skewed, so a small cache covers a large share of traffic, and prefixes are short and identical across users. Caching at the edge removes most requests from your infrastructure entirely. Personalisation breaks this, because a personalised response is unique per user, which is the real argument for keeping personalisation as a light reordering of a cacheable base result.
Rebuild and swap, never mutate in place
Updating a live trie under query load means locking or accepting inconsistent reads. Building a new one and swapping the pointer makes updates atomic and instant, and lets you validate the new index before it serves anything. It costs double the memory during the swap, which is the price of never serving a half-updated index.
Debounce on the client, because most keystrokes are noise
Firing a request per character means most requests are for prefixes the user is typing through and will never see results for. A short debounce and cancelling in-flight requests removes the majority of load before it exists. It adds perceived latency equal to the debounce, so this is tuned in tens of milliseconds and is worth measuring rather than guessing.
How it changes with scale
Request volume is roughly typing speed times users, so it is very high but also very cacheable and very uniform. The trie's memory grows with vocabulary rather than traffic, which means the index is often small enough to sit in memory on every node, removing a network hop from the critical path entirely.
Where it breaks first
A stale or failed index build. Because the serving path keeps working with the previous index, nothing errors and latency is unaffected; suggestions simply stop reflecting reality. The only way this gets noticed is by monitoring build freshness explicitly, which is the kind of alert that exists only after the first time it happens.
Draw this yourself
Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.
When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.
Questions about this design
Trie or a search engine?
A trie when the requirement is genuinely prefix completion, because it is faster and simpler. A search engine when you need fuzzy matching, typo tolerance and relevance signals, which most product searches eventually do.
How do you handle typos?
Edit-distance expansion at query time, or by indexing common misspellings from the query log. The log-based approach is usually better because it captures the typos your users actually make rather than every possible one.
Should suggestions be personalised?
Lightly. Recent personal searches ranked above global ones covers most of the benefit while keeping the base result cacheable. Fully personalised suggestions destroy your cache hit rate for a small quality gain.
How do you keep offensive suggestions out?
A blocklist applied at build time rather than at query time, so the filtering cost is paid once per rebuild rather than on every request. It also means the filter is auditable, since you can inspect the index.
More templates
- File Storage Service DesignThe design goal is to keep file bytes away from your servers entirely, so that what you run is a metadata service that happens to be about files.
- Ticket Booking System DesignA named seat can be sold exactly once, and ten thousand people want it at the same instant. Everything here follows from that.
- Food Delivery System DesignThree sides to coordinate and one deadline that nobody controls: the kitchen. Dispatch quality matters more here than any amount of throughput.
- Multiplayer Game Backend DesignLatency is the product. Every architectural decision here is subordinate to keeping the round trip between a player and the simulation short.
Last updated