References

References

The literature harness-kit is built on, and what each source actually changed here. Sources that only confirmed what we already did are not worth listing; these are the ones that cost us a commit.

The harness

Böckeler, Harness engineering for coding agent users (martinfowler.com). The foundation. Guides are feedforward, they steer the agent before it acts. Sensors are feedback, they observe after and let it self-correct. Both come in two execution types: computational (deterministic, CPU-run, cheap enough for every change) and inferential (needs a model's judgment). The whole pipeline is this shape, and Execution: computational | inferential on every sensor is this taxonomy made literal.

Böckeler, Maintainability sensors for coding agents (martinfowler.com). The follow-up, and the most expensive article on this page.

Three findings landed directly in the code:

1. "The agent reliably ignores sensor checks unless hardwired via hooks or extensions", and markdown guides alone are "quite unreliable". code-conventions.md was a markdown file politely asking the agent to run the linter. code-maintainability now runs it.
2. The warning about "a false sense of security and an illusion of quality". We had built exactly that: sensors declaring themselves deterministic hard gates while checking nothing, and a quality log recording them as passed on every run. Hence exit 3 (inferential, never a pass) and exit 2 (a computational sensor that wires up no check is broken, not passing).
3. The limits that bound complexity, max function arguments, file length, function length, cyclomatic complexity, "weren't even active in ESLint's default preset, I had to configure maximums for them first". Same is true of ruff. They are configured in pyproject.toml now, and CI runs them. That immediately found real debt.

Her warning about feedback overload, "sending it into a spiral of over-engineered refactorings", is why the one complexity violation we found is a named per-file ignore with the reason written down, rather than a refactor of untested code to satisfy a rule added yesterday.

Fowler, Agentic Programming (martinfowler.com). Names the shift the pipeline assumes: humans stop typing code and start reviewing it, "still responsible for what the software does" but through "code review, examining test results, and reviewing outputs from other sensors". Harness engineering is the central skill. This is the argument for why the two human gates are where they are.

Evals and the judge

Husain, Using LLM-as-a-Judge for evaluation and Your AI product needs evals. The critique that lands on our rubrics: uncalibrated 1-10 scales get interpreted differently by every grader, binary judgments are more reliable, and the thing that actually matters is measured agreement between the judge and human labels. harness-kit does not do that yet. Our 8.0 threshold is a convention, not a calibrated boundary, and pipeline-pattern.md says so out loud rather than implying the score is a measurement.

Panickssery et al., Self-Preference Bias in LLM-as-a-Judge (arXiv). LLM judges systematically prefer text from their own family, tracking perplexity: they rate what is familiar to them higher than humans do. Dispatching a fresh evaluator, which we do, removes the author's context but not this. A fresh Claude judging Claude's prose is still self-preference. An honest mitigation is a different model family or human labels; we have neither yet, and the wiki should not pretend otherwise.

These two together are why eval-score.py exists. It cannot make the judgment trustworthy, but the arithmetic was never a judgment call: it reads the weights out of the rubric and recomputes the total, so at minimum the number follows from the scores.

Testing the harness itself

No citation needed for the biggest lesson. harness-kit gated every artifact it produced and had nothing gating itself: no tests, no CI, no linter. That is how a one-word heading mismatch disabled ~30 section assertions across three sensors and nobody noticed. The test that would have caught it on day zero is four lines, and it is now generic over every sensor, including ones not written yet.

The harness needs a harness.

The engineering canon (system-architect)

The system design method draws on Kleppmann (Designing Data-Intensive Applications), Jeff Dean's latency numbers, Vogels on eventual consistency, Helland on data on the outside vs inside, Nygard (Release It!) on stability patterns, and Ousterhout (A Philosophy of Software Design) on complexity. Those inform the design rubric and the ten staff questions rather than the harness mechanics.

See also

• Harness Engineering: the theory, in depth
• Sensors and Evals: the two feedback halves as implemented
• Agent Pipelines: where each gate sits per agent