Payment system architecture.
The only system on this list where being approximately right is indistinguishable from being wrong, and where the provider, not you, holds the truth.
The components
Every row below is read from the graph that produced the diagram above, so the two cannot disagree.
| Component | Tier | Why it is there |
|---|---|---|
| Checkout | Client | Supporting component |
| API gateway | Edge | Supporting component |
| Payment intent | Application | Idempotency key required on every write |
| Ledger | Application | Double entry, append only |
| Risk & fraud | Application | Supporting component |
| Reconciliation | Application | Compares our ledger to the provider daily |
| Webhook handler | Application | Provider is the source of truth for capture |
| Postgres | Data | Serializable for ledger writes |
| Outbox | Data | Transactional, so events cannot be lost |
| Kafka | Data | Supporting component |
| Payment provider | External | Supporting component |
| Banking rails | External | Supporting component |
Design decisions worth arguing about
A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.
Idempotency keys are not optional
Networks retry, users double-click, and clients time out on requests the server actually completed. Without a caller-supplied idempotency key stored alongside the result, every one of those becomes a duplicate charge. The design cost is real: you have to store keys, decide how long to keep them, and define what happens when the same key arrives with a different payload, which should be an error rather than a silent overwrite.
Double entry is a constraint, not an accounting style
Recording every movement as balanced debits and credits on an append-only ledger means the books can be checked by summing them, and a bug that loses money shows up as an imbalance rather than as a number that is quietly wrong. It costs more rows and more discipline: no updates, no deletes, and corrections expressed as new compensating entries rather than edits to history.
The provider is the source of truth for capture
You can record an intent, but you cannot know a payment succeeded until the provider says so, and the provider says so by webhook, which can arrive late, out of order, or twice. Treating your own optimistic state as authoritative is how systems end up shipping goods for payments that later fail. The consequence is that webhook handling is a first-class part of the design rather than an afterthought, and it needs the same idempotency treatment as the API.
A transactional outbox instead of dual writes
Writing to the database and then publishing to a broker is two operations with no shared transaction, so a crash between them loses the event permanently. Writing the event into an outbox table inside the same transaction as the ledger entry, and relaying it separately, makes the publish exactly as durable as the write. The price is a relay component to run and monitor, plus at-least-once delivery downstream, which pushes idempotency onto every consumer.
Reconciliation is a feature, not a safety net
Even with all of the above, your ledger and the provider's will diverge, through partial failures, disputes, refunds processed out of band. A daily job comparing the two and flagging differences is the only mechanism that actually catches this, and the important design decision is what it does when it finds something: alerting a human is correct, auto-correcting is usually not.
How it changes with scale
Payment volume is small next to most systems on this list, so throughput is rarely the constraint. What grows painfully is the ledger, because it is append-only and never deleted, and the reconciliation window, because comparing two growing datasets daily gets slower. Partitioning the ledger by period is the usual answer and it should be planned before it is needed, since migrating an append-only table under load is unpleasant.
Where it breaks first
Webhook delivery gaps. Providers retry, but not forever, and a handler that is down or returning errors during that window loses the notification permanently. The system then believes payments are pending that have long since succeeded. This is exactly what reconciliation is for, which is why a design that treats it as optional fails silently and slowly rather than loudly and fast.
Draw this yourself
Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.
When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.
Questions about this design
More templates
- URL Shortener System DesignThe canonical read-heavy system: roughly a hundred reads for every write, and a redirect that has to be fast enough that nobody notices it happened.
- Chat Application System DesignLong-lived connections change everything: the hard part is not storing messages, it is knowing which of your servers is holding the socket you need to write to.
- RAG Pipeline ArchitectureAlmost every RAG system that disappoints is failing at retrieval, not generation, and the architecture is what decides whether you can tell.
- E-commerce Platform ArchitectureFour services with four completely different consistency requirements, which is the whole reason this is not one application.