File storage service architecture.
The design goal is to keep file bytes away from your servers entirely, so that what you run is a metadata service that happens to be about files.
The components
Every row below is read from the graph that produced the diagram above, so the two cannot disagree.
| Component | Tier | Why it is there |
|---|---|---|
| Client | Client | Uploads direct to storage, not through the API |
| CDN | Edge | Supporting component |
| API gateway | Edge | Supporting component |
| Metadata service | Application | Supporting component |
| URL signer | Application | Presigned PUT keeps bytes off our servers |
| Chunker | Application | Content-defined chunks enable dedup |
| Virus scan | Application | Supporting component |
| Sync service | Application | Delta sync, not whole-file re-upload |
| Postgres | Data | Supporting component |
| Block store | Data | Supporting component |
| Kafka | Data | Supporting component |
Design decisions worth arguing about
A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.
Presigned uploads, so bytes never traverse your API
Proxying uploads means your API's bandwidth, memory and timeouts all scale with file size, and a few large uploads can starve every other request. Issuing a presigned URL and letting the client write directly to object storage removes that entirely. The cost is that you no longer observe the upload, so completion has to be confirmed by the client or by a storage event, and you must handle objects that were uploaded but never confirmed.
Content-defined chunking, not fixed-size blocks
Splitting at fixed offsets means inserting a byte near the start shifts every subsequent block and defeats deduplication completely. Choosing boundaries from the content itself keeps chunks stable across edits, so a small change re-uploads a small amount. It costs more CPU to compute boundaries and produces variable-size chunks, which complicates storage accounting.
Deduplication is a privacy decision as much as a storage one
Deduplicating across all users saves a great deal of storage and leaks information: upload timing reveals whether a chunk already existed, which tells you whether someone else has the same file. Deduplicating only within an account avoids that and saves much less. This is a decision that should be made deliberately, because it is very hard to reverse once the store is built.
Delta sync, because whole-file transfer does not scale with edits
Re-uploading an entire document because one paragraph changed is what makes sync feel slow on large files. Transferring only changed chunks makes edit cost proportional to the edit. It requires both ends to agree on chunk boundaries and to maintain an index of them, which is real complexity and the main reason simple sync tools do not do it.
How it changes with scale
Metadata operations vastly outnumber byte transfers, since listing, syncing and permission checks happen constantly while uploads are comparatively rare. That means the database is the component under pressure, not the storage layer, which is the opposite of what the name of the service suggests.
Where it breaks first
A sync loop. Two clients with slightly different views of the same file each write what they believe is the correct version, and each write triggers the other to sync again. Without version vectors and a conflict resolution rule that produces a stable outcome, this consumes bandwidth indefinitely and users watch a file change under them.
Draw this yourself
Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.
When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.
Questions about this design
More templates
- URL Shortener System DesignThe canonical read-heavy system: roughly a hundred reads for every write, and a redirect that has to be fast enough that nobody notices it happened.
- Chat Application System DesignLong-lived connections change everything: the hard part is not storing messages, it is knowing which of your servers is holding the socket you need to write to.
- Payment System DesignThe only system on this list where being approximately right is indistinguishable from being wrong, and where the provider, not you, holds the truth.
- RAG Pipeline ArchitectureAlmost every RAG system that disappoints is failing at retrieval, not generation, and the architecture is what decides whether you can tell.