-
Notifications
You must be signed in to change notification settings - Fork 0
Sensors
Sensors are the computational, deterministic feedback control. They observe the artifact after the agent writes it and return a hard pass/fail. Same artifact, same verdict, every time. A failing sensor blocks approval and the agent regenerates until it passes.
A sensor is a plain markdown file describing checks, plus a small Python runner that applies them with real regex. The markdown is the spec; the runner is the enforcement. Example shape:
# Sensor: PRD Structure
Type: deterministic
Mode: hard gate
## Required sections (all present, in order)
- Problem and Hypothesis
- Customers
- ...
## Forbidden tokens
- lorem, TODO, FIXME, placeholder
## Markdown rules
- exactly 1 H1 heading
- no em-dash
- no ASCII box-drawing
## On failure
Block publish. Return missing sections, forbidden tokens, rule violations.
Agent regenerates failed parts only.sensor-runner.py parses known sections out of the sensor markdown and checks them against the
artifact:
-
Required sections: for each bullet, it builds a heading regex (tolerant of numbered prefixes
like
## 3) ...) and fails if the heading is absent. -
Forbidden tokens: fails if any appear (catches
TODO,lorem, unfilled template tokens like{System Name}). - Markdown rules: exactly one H1, no em-dash, no ASCII box-drawing, minimum mermaid blocks, etc.
It exits 0 if everything passes and 1 if any blocking check fails, printing the specific failures.
A PostToolUse hook fires it on save; on failure it returns the feedback to the agent, which fixes only
the failed parts.
Structure is not a matter of opinion, so it should not cost an LLM call or be subject to a model's mood. Pushing every checkable rule into a sensor:
- makes the gate fast and free (no tokens),
- makes it exact (no false "looks fine"),
- frees the eval to judge only what genuinely needs semantic judgment.
This is the core harness-engineering move: make deterministic what you can, infer only what you must.
Each stage has its own sensors. A sample:
| Stage | Sensors |
|---|---|
prd |
prd-structure, prd-acceptance-criteria
|
prp |
prp-structure, prp-context-quality, prp-links, link-validator
|
plan |
plan-structure |
dev |
code-conventions, test-coverage, dev-structure
|
| system design |
design-structure (sections, mermaid, no unfilled tokens), design-rigor (numbers, sizing math, trade-offs, phases) |
design-rigor is a good example of a sensor enforcing more than shape: it fails an SDD that has no
numeric targets, no back-of-envelope math, fewer than three trade-offs, or no vertical-slice phase,
all deterministically detectable, all things you want to force without burning an eval on them.
A sensor's value is not just the verdict, it is the message. "missing required section: 'Success Metrics'" tells the agent exactly what to add. Good sensor output reads like instructions for the next turn, not a stack trace. This is what makes deterministic feedback a self-correction loop instead of a wall.
- Keep checks objective: if a human could disagree, it belongs in an eval, not a sensor.
- Name the failure precisely so the fix is obvious.
- Use forbidden tokens to catch unfilled template placeholders (
{N},{...},TBD). - Mark it hard gate; sensors do not have a threshold, they pass or block.
- Evals: the inferential half that judges meaning
- Harness Engineering: computational vs inferential controls
- Böckeler, Maintainability sensors for coding agents, martinfowler.com, 2026
The harness
- Harness Engineering
- References
- Guides
- Sensors
- Evals
- Pipeline and Stages
- Golden Path
- Agents
- Agent Pipelines
- Designer Skill
v5: autonomy
System design