Scripts Not Instructions

Scripts, not instructions

Sections 4 to 4a of the method. Mechanical checks live in scripts. Reasoning an agent does twice becomes a script.

4. Scripts, not instructions

These scripts do what I used to write into every agent brief. The two under scripts/ are my API repository's own; the rest ship in this repository's bin/, each with a --self-test:

• scripts/preflight.sh builds, runs the named test classes, checks the changelog and module versions, restores in locked mode before building so a stale lock file cannot pass, fails when a test filter matches zero tests, scans added lines including untracked files for house style, and parses any changed workflow file with a duplicate-key-rejecting parser.
• scripts/sync-master.sh merges the default branch, regenerates lock files when a project file changed, and reports conflicts.
• needs-review.sh, in this repository, prints the rules above, read from .lean/needs-review.rules. A missing rules file or merge base prints a line too, so a check that could not run never looks like a quiet diff.
• asset-provenance.py, in this repository, finds embedded provenance metadata in images and removes it with --strip. Section 9 is why.
• interaction-surface.sh, in this repository, prints what a diff touches that it did not write. Section 13 is why.
• agent-hygiene.sh, in this repository, reports what parallel agents leave behind: stuck wait loops, orphaned servers, scratchpad collisions, worktrees on merged branches. Section 14a is why.
• retro-signals.sh, in this repository, collects the facts a retro starts from. Section 16 is why.
• bump-version.sh, in this repository, moves a Node package's version including the two lock file fields that belong to it and none that do not. Three bumps in one day went wrong in three different ways, which is one more than care can be expected to cover.
• heavy.sh, in this repository, queues heavy commands per lane with a deadline on the wait (HEAVY_WAIT, default 300 seconds, then exit 75), and names the processes holding the lane. Section 11 is why.
• pr-blockers.sh <pr>, in this repository, lists every reason a pull request cannot merge in one read-only call: failing or missing required checks, runs waiting for fork approval, unresolved threads, requested changes, ruleset rules, the head author's identity, how far behind it is, and which gh pr merge form the base accepts. A hook points at it when gh pr merge fails.
• assert-ran.sh, in this repository, reads the executed test count from a runner's JSON, trx, go test -json or JUnit output and fails at zero or below --min. Section 10 is why.
• why-red.sh <log>, in this repository, names environmental causes in a failed log (no Docker, a port in use, the inotify limit, wrong architecture, wrong Node, a full disk, DNS) before anyone debugs the code.
• check-conflict-markers.sh, check-top-level-md.sh and check-workflow-keys.py, in this repository, fail on conflict markers in tracked or untracked files, on a new root Markdown file that is not on an allowlist (pull request bodies get committed this way), and on duplicate keys in a changed workflow file.
• sync-default.sh, in this repository, refuses a dirty tree, merges the default branch, lists conflicts, then runs a locked restore for each lock file present so a stale lock file fails here instead of in CI. It never resets, stashes or checks out files.
• check-deployed-sha.sh <url> <sha>, in this repository, proves a deploy by the commit sha the running build reports, not a 200 or a version string. Section 14e is why.
• check-runtime.sh, in this repository, compares the Node, .NET and Go versions a repository declares with the binaries on PATH.

The general parts of the two scripts/ entries now ship here as sync-default.sh, check-workflow-keys.py, assert-ran.sh and style-scan. For the rest, copy the shape and replace the checks with your own.

One trap worth inheriting along with the shape: needs-review.sh diffs the working tree against the merge base, folding untracked files in as new. That is deliberate, because a rule that only reads committed changes misses the file an agent has written and not yet added. It also means running it in a checkout full of other work walks all of that too. Run it inside a clean copy of the branch. I lost a few minutes to this before writing it down.

An instruction in a prompt is forgotten within a day. A script is not, and it costs no tokens to obey.

4a. The second time an agent reasons its way through something, it becomes a script

Section 4 moved the checks I kept writing into briefs into scripts. The same rule goes further: anything an agent works out by reasoning, and then needs again, gets written down as something that runs.

The first time, the agent spends tokens understanding the problem: how to land a stack of pull requests safely, how to cut a site over with a rollback, how to run a workflow's jobs without the hosted runner. The second time, if nothing was written down, it spends them again, and gets a slightly different answer. Written as a script, the next agent runs one command, waits, and reads the result. The tokens are spent once, on writing it.

flowchart LR
    T[A task comes up] --> Q{"Done before?"}
    Q -- no --> R[The agent reasons it through]
    R --> W{"Will it come up again?"}
    W -- no --> Done[Done]
    W -- yes --> S["Write it as a script,<br/>with a test that fails when it is wrong"]
    S --> Done
    Q -- yes --> X[Run the script and wait for the result]
    X --> V{"Did it do the job?"}
    V -- yes --> Done
    V -- no --> F[Fix the script, not the run]
    F --> X

Loading

Where the script goes decides who benefits. A script that works on any repository (landing a stack of pull requests, collecting retro signals, a wait with a deadline, the hygiene sweep) goes to this repository as a pull request, with its test and a line in section 4's list, and every project uses it from here. A script that knows one repository's layout or rules (its preflight, a site's cutover) goes to that repository's scripts/; if part of it is general, that part comes here too. Either way it arrives as a pull request, gets the same adversarial review as any change, and a person merges it. Nothing is pushed straight to the default branch of this repository or any other.

Three rules keep it honest:

• The script gets a test. A script that passes having done nothing is worse than a brief, because it looks finished. The stack-landing script in 14d stopped the landing twice for a reason that turned out to be its own bug, and the local runner in 14g got three things wrong until it had tests.
• Fix the script, not the run. When a script is wrong, correcting its output by hand this once means the next agent meets the same bug. Change the script and run it again.
• It stops at the first surprise and says so. A script cannot judge what it was not written for, so it checks its assumptions, stops when one fails, and names the failure. Deciding what to do then is the part that stays with a person or an agent.

A rule about the code's shape becomes a test. The same goes for the rules in a repository's agent instructions. Any rule that can be stated about types, namespaces or references (the core never references a module, endpoints stay internal, every list endpoint takes a page size, no controllers) can be an architecture test: reflection over the built assembly, or a library such as NetArchTest or ArchUnit. The agent does not have to remember it, the build fails when it is broken, and the instructions file names the test instead of restating the rule. Like any check, it first asserts it found something to check, because a scan of the wrong assembly passes every rule.

This is what site reliability engineering calls eliminating toil, and what operations people call a runbook written as code. With agents it matters more, because the reasoning is the expensive part, and it is exactly the part a script lets you skip.

Examples from one day: landing a stack of seven pull requests (14d), a production cutover with backups, confirmations and a tested rollback (14f), running a repository's workflow locally when its CI minutes ran out (14g), and the holdout and preflight checks that run on every change (2, 4).

---

Previous: The diff decides whether a critic runs | Home | Next: Rework and concurrency