System design template

CI/CD pipeline architecture.

Two things decide whether people trust a pipeline: how fast it is, and whether a red build actually means something is broken.

Download for macOS
v0.1.33 · .dmg · Apple Silicon & Intel
Signed & notarized by Apple · opens without a Gatekeeper warning
sha256 698955a0187bc039f4c74f5d05a9f10fbb27376a45788a0a241d1326b73873c7
Download for Windows instead
$curl -fsSL https://lucidtrain.com/install.sh | sh
CI/CD pipeline architecture. 12 components across 5 tiers.
CI/CD pipeline architecture. Rendered by the same ELK layout engine the app runs: the agent emits components, tiers and edges, and the engine places them, so the boxes cannot overlap.

The components

Every row below is read from the graph that produced the diagram above, so the two cannot disagree.

ComponentTierWhy it is there
DeveloperClientSupporting component
GitHubExternalSupporting component
Webhook receiverApplicationSupporting component
Job schedulerApplicationFans a pipeline into a job DAG
RunnersApplicationEphemeral, so no state leaks between builds
Test stageApplicationSupporting component
Security scanApplicationSupporting component
DeployerApplicationProgressive rollout, automatic rollback
Build cacheDataThe single biggest lever on build time
Image registryDataSupporting component
Artifact storeDataSupporting component
Production clusterInfrastructureSupporting component

Design decisions worth arguing about

A diagram shows what was chosen. It does not show what it cost, and that is usually the part that matters in a review or an interview.

Ephemeral runners, so builds cannot contaminate each other

Long-lived runners accumulate state: leftover dependencies, files from previous jobs, a cache that is subtly wrong. That state produces builds that pass on one runner and fail on another, which is the most expensive kind of flakiness because it looks like a code problem. Fresh containers eliminate it, at the cost of losing everything a warm machine had, which is exactly what makes an external cache mandatory rather than optional.

The cache is the main lever on build time

Once runners are ephemeral, most of a build is re-downloading and recompiling things that have not changed. A shared cache keyed on a lockfile hash usually cuts build time more than any amount of parallelism. The risk is correctness: a cache key that is too coarse serves stale artifacts and produces builds that succeed against dependencies you are not actually shipping, which is a genuinely dangerous failure because nothing looks wrong.

Fail fast, but only on signal

Ordering cheap checks before expensive ones gives faster feedback and cheaper failures. The tension is with flaky tests: a pipeline that fails randomly trains people to rerun without reading, and once that habit forms the pipeline has stopped being a safety mechanism regardless of its coverage. Quarantining flaky tests aggressively is worth more than adding new ones.

Progressive rollout so a bad deploy is bounded

Deploying everywhere at once means the blast radius of a bad release is everything. Rolling out to a fraction while watching error rates bounds it, and automatic rollback bounds the duration. The cost is that a deploy takes longer and you must run two versions simultaneously, which forces backward-compatible database migrations as a standing discipline rather than a special case.

How it changes with scale

Queue time, not build time, is what people experience as a slow pipeline, and it is a function of concurrency limits rather than of any individual job. Monorepos change the problem: without change detection, every commit builds everything, and the pipeline gets slower as the repository grows regardless of what was actually touched.

Where it breaks first

Cache poisoning. A corrupt or mis-keyed cache entry gets reused across builds, and the resulting failures look like code problems in unrelated places. Because the cache is invisible in the logs, this is usually diagnosed only after someone thinks to clear it, which is why cache keys should be conservative and manual invalidation should be a one-click operation.

Draw this yourself

Open the Diagram tab and describe the system. The agent emits a semantic graph rather than coordinates, so you can edit the components and the layout re-solves instead of drifting.

shell
$ Diagram a CI/CD pipeline: GitHub webhooks into a job scheduler, ephemeral container runners with a shared build cache, test and security scan stages, an image registry, artifact storage, and a deployer doing progressive rollout to a Kubernetes cluster.

When the shape is right, Implement in code turns the canvas into a markdown specification, every component, every relationship and the notes, and starts a real turn in the Code tab with it.

FAQ

Questions about this design

Should deployment be in the same pipeline as CI?

Same definition, separate stages with an explicit boundary. Merging them entirely makes every test run a potential deploy; separating them completely tends to produce a deploy process nobody exercises until it is urgent.

How do you keep secrets out of builds?

Short-lived credentials issued to the job by an identity provider rather than long-lived secrets in environment variables. Ephemeral runners help, since there is no persistent machine left holding anything afterwards.

Where should security scanning happen?

In the pipeline for fast feedback, and continuously against the registry afterwards, because a vulnerability disclosed tomorrow applies to an image that passed today. Scanning only at build time means your view of risk is as old as your last deploy.

Is a self-hosted runner worth it?

For heavy builds or work needing private network access, yes, and it usually pays for itself. It also brings back the maintenance and the isolation problems that hosted runners were solving, so the saving is real but not free.

More templates