Harness engineering

Harness design patterns, across three vocabularies.

Three labs, three vocabularies, one shared concept between them. Here is the mapping.

Download for macOS
v0.1.33 · .dmg · Apple Silicon & Intel
Signed & notarized by Apple · opens without a Gatekeeper warning
sha256 698955a0187bc039f4c74f5d05a9f10fbb27376a45788a0a241d1326b73873c7
Download for Windows instead
$curl -fsSL https://lucidtrain.com/install.sh | sh

The three most authoritative sources on harness engineering use almost entirely different vocabularies for overlapping ideas. Böckeler on martinfowler.com writes in guides and sensors, OpenAI in agent legibility and invariants, Anthropic in generators, evaluators and context resets. A term-frequency check across all three full texts finds feedback loop is the only concept present in every one. This page maps them onto each other.

Why a reconciliation is needed

Reading all three primary sources back to back is disorienting, because they describe recognisably similar practice in words that do not overlap. Feedforward and sensors appear dozens of times in Böckeler and never in the other two. Evaluator appears throughout Anthropic's write-up and in neither of the others. Legibility and invariant are OpenAI's and nobody else's. Only feedback loop is common to all three, and even then only a couple of times each.

Steering before the act

Böckeler calls these guides, feedforward controls that anticipate behaviour and steer before the agent acts. OpenAI's version is progressive disclosure and agent legibility: give the agent a map rather than a thousand-page manual, on the principle that anything it cannot access in context effectively does not exist. Anthropic's sprint contract, where a generator and evaluator agree what done means before code is written, is the same idea applied to acceptance rather than to conventions.

Checking after the act

Böckeler calls these sensors, and notes they are most powerful when their output is written for a model to consume, such as a linter message carrying its own correction instructions. Anthropic's equivalent is the evaluator, separated from the generator specifically because of what they call the self-evaluation problem: agents asked to assess their own output tend to praise it confidently even when the quality is obviously mediocre. OpenAI's is enforcing invariants rather than implementations, letting agents ship fast without undermining foundations.

Managing the context window

Anthropic is the most developed here and distinguishes a context reset from compaction: clearing the window entirely and starting a fresh agent with a structured handoff, rather than summarising in place. Their reason is specific, that compaction does not give a clean slate so context anxiety persists. OpenAI's progressive disclosure attacks the same problem earlier, by not filling the window in the first place. Böckeler treats it as a property of a good guide.

The principle all three would sign

Anthropic states it most directly: every component in a harness encodes an assumption about what the model cannot do on its own, and those assumptions are worth stress testing, both because they may be wrong and because they go stale as models improve. Their own article documents removing the sprint construct and context resets on a newer model that no longer needed them. Böckeler's closing line points the same way, that building the outer harness is an ongoing engineering practice rather than a one-time configuration.

How Lucid Train maps onto it

Guides: Claude Code-compatible skill packs, task personas, a repository map under a token budget, and a diagram used as an implementation specification. Sensors: reviewable diffs before anything is applied, a dependency audit, and a publish-state guard that blocks destructive commands once an acceptance check has passed. Context: automatic compaction with an emergency retry on provider overflow, plus persistent offline memory. Isolation: dispatched sub-agents run in separate git worktrees with the diff applied back.

The same ideas, three vocabularies

IdeaFowler / BöckelerOpenAIAnthropic
Steer before actingGuides, feedforwardProgressive disclosure, agent legibilitySprint contract
Check after actingSensors, feedbackEnforce invariants, not implementationsEvaluator, separate from generator
Fast deterministic checksComputational controlsGolden principlesNot named
Model-judged checksInferential controlsNot namedEvaluator, LLM as judge
Context managementA property of good guidesProgressive disclosureContext reset vs compaction
Failure near the limitNot namedNot namedContext anxiety
Self-assessment is unreliableNot namedNot namedSelf-evaluation problem
Shared by all threeFeedback loopFeedback loopFeedback loop
FAQ

Questions

Which harness source should I read first?

Böckeler's article on martinfowler.com is the most complete treatment of what belongs in a harness. Anthropic's is the best on context management and on why an agent cannot evaluate its own work. OpenAI's is the best account of running this at scale.

What is a cybernetic governor in this context?

Böckeler's description of what a harness is: something combining feedforward and feedback to regulate the codebase toward a desired state, in the control-theory sense rather than as a metaphor.

What is context anxiety?

Anthropic's term for a failure mode where a model begins wrapping up work prematurely as it approaches what it believes is its context limit. They distinguish a context reset from compaction partly to address it, since compaction leaves the agent without a clean slate.

Why do the three sources disagree on vocabulary?

Because the practice arrived before the terminology. All three were published within about fourteen months of each other by teams working independently, and no standards process has run over it.

Related