API gateway architecture.
One entry point, so authentication, rate limiting and observability are implemented once instead of in every service.
The components
Every row below is read from the graph that produced the diagram above, so the two cannot disagree.
| Component | Tier | Why it is there |
|---|---|---|
| Clients | Client | Supporting component |
| CDN | Edge | Supporting component |
| API gateway | Edge | One entry point, one place for cross-cutting concerns |
| Auth | Application | Verified once at the edge, not per service |
| Rate limiter | Application | Supporting component |
| Router | Application | Path and header based, config not code |
| Orders service | Application | Supporting component |
| Catalog service | Application | Supporting component |
| Aggregator | Application | One client call, several upstreams |
| Redis | Data | Supporting component |
| Postgres | Data | Supporting component |
| Prometheus | Infrastructure | Per-route latency and error rate |
Design decisions worth arguing about
A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.
Cross-cutting concerns belong at the edge, once
Authentication, rate limiting, request logging and CORS are needed by every service, and implementing them per service means implementing them slightly differently per service. Verifying the token once at the gateway and passing an identity downstream means the services trust the edge and can stay simple. The cost is that the gateway becomes a component every request depends on, so its availability is the system's availability.
Aggregation helps the client and hides the coupling
Letting the gateway call three services and combine the result saves a mobile client three round trips over a slow network, which is a real win. It also puts knowledge of those three services into the gateway, so a change to any of them can break it. Aggregation logic tends to accumulate until the gateway is a service with its own release cycle, and the honest response is to notice when that has happened rather than to pretend it has not.
Routing as configuration, not code
Expressing routes as configuration means adding a service is a config change rather than a gateway deploy, which matters when many teams ship independently. It also means a routing mistake is a config mistake with no compiler to catch it, so the configuration needs validation and a staged rollout of its own.
The gateway must not become a service mesh
North-south traffic, from outside to inside, is the gateway's job. East-west traffic between internal services is not, and routing it through the gateway adds a hop and a shared failure point to every internal call. The line blurs gradually and is worth defending explicitly.
How it changes with scale
The gateway scales horizontally because it is stateless, so the interesting limits are elsewhere: shared rate-limit state, and the connection pool to each upstream. Per-route metrics become the operational surface, since an aggregate error rate on the gateway hides which upstream is actually failing.
Where it breaks first
The gateway becoming the single point of failure it was designed to consolidate. Every request passes through it, so a bad config push or a memory leak takes down services that are individually healthy. This is the argument for staged config rollout and for the edge proxy in front being able to serve a useful error rather than a connection refusal.
Draw this yourself
Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.
When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.
Questions about this design
More templates
- URL Shortener System DesignThe canonical read-heavy system: roughly a hundred reads for every write, and a redirect that has to be fast enough that nobody notices it happened.
- Chat Application System DesignLong-lived connections change everything: the hard part is not storing messages, it is knowing which of your servers is holding the socket you need to write to.
- Payment System DesignThe only system on this list where being approximately right is indistinguishable from being wrong, and where the provider, not you, holds the truth.
- RAG Pipeline ArchitectureAlmost every RAG system that disappoints is failing at retrieval, not generation, and the architecture is what decides whether you can tell.