API gateway architecture.
One entry point, so authentication, rate limiting and observability are implemented once instead of in every service.
The components
Every row below is read from the graph that produced the diagram above, so the two cannot disagree.
| Component | Tier | Why it is there |
|---|---|---|
| Clients | Client | Supporting component |
| CDN | Edge | Supporting component |
| API gateway | Edge | One entry point, one place for cross-cutting concerns |
| Auth | Application | Verified once at the edge, not per service |
| Rate limiter | Application | Supporting component |
| Router | Application | Path and header based, config not code |
| Orders service | Application | Supporting component |
| Catalog service | Application | Supporting component |
| Aggregator | Application | One client call, several upstreams |
| Redis | Data | Supporting component |
| Postgres | Data | Supporting component |
| Prometheus | Infrastructure | Per-route latency and error rate |
Design decisions worth arguing about
A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.
Cross-cutting concerns belong at the edge, once
Authentication, rate limiting, request logging and CORS are needed by every service, and implementing them per service means implementing them slightly differently per service. Verifying the token once at the gateway and passing an identity downstream means the services trust the edge and can stay simple. The cost is that the gateway becomes a component every request depends on, so its availability is the system's availability.
Aggregation helps the client and hides the coupling
Letting the gateway call three services and combine the result saves a mobile client three round trips over a slow network, which is a real win. It also puts knowledge of those three services into the gateway, so a change to any of them can break it. Aggregation logic tends to accumulate until the gateway is a service with its own release cycle, and the honest response is to notice when that has happened rather than to pretend it has not.
Routing as configuration, not code
Expressing routes as configuration means adding a service is a config change rather than a gateway deploy, which matters when many teams ship independently. It also means a routing mistake is a config mistake with no compiler to catch it, so the configuration needs validation and a staged rollout of its own.
The gateway must not become a service mesh
North-south traffic, from outside to inside, is the gateway's job. East-west traffic between internal services is not, and routing it through the gateway adds a hop and a shared failure point to every internal call. The line blurs gradually and is worth defending explicitly.
How it changes with scale
The gateway scales horizontally because it is stateless, so the interesting limits are elsewhere: shared rate-limit state, and the connection pool to each upstream. Per-route metrics become the operational surface, since an aggregate error rate on the gateway hides which upstream is actually failing.
Where it breaks first
The gateway becoming the single point of failure it was designed to consolidate. Every request passes through it, so a bad config push or a memory leak takes down services that are individually healthy. This is the argument for staged config rollout and for the edge proxy in front being able to serve a useful error rather than a connection refusal.
Draw this yourself
Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.
When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.
Questions about this design
Do I need an API gateway for a small system?
For one or two services, no. The value appears when several services need the same cross-cutting behaviour and you are tired of implementing it repeatedly.
Gateway or service mesh?
A gateway handles traffic entering the system; a mesh handles traffic between internal services. They solve different problems and larger systems often run both.
Should the gateway do authorisation as well as authentication?
Authentication yes, since verifying a token once is strictly better than doing it repeatedly. Authorisation usually no: whether a user may act on a specific resource depends on domain state the gateway does not have.
Where should rate limit state live?
A shared store for accuracy, with a local counter absorbing most checks. The tradeoff is a round trip on every request versus allowing slightly more through than the limit strictly permits.
More templates
- Feature Flag Service DesignEvaluation has to be local and instant, because a flag check sits in the hot path of code that would otherwise not make a network call at all.
- Webhook Delivery System DesignYou are making requests to servers you do not control, which are frequently slow, sometimes wrong, and occasionally gone.
- Audit Logging System DesignAn audit log that can be edited is not an audit log, which makes this one of the few systems where the absence of features is the design.
- Vector Search Service DesignThe failure mode here is not an error. It is recall dropping slowly as the index grows, with nothing in your metrics saying so.
Last updated