Interaction Surface

Every finding is an interaction

Section 13 of the method. Look at what the diff touches but did not write, before you push.

In one run in September 2026, agent-made pull requests across three repositories went through the critic and a review bot. Over the first five, every defect that survived verification was an interaction between two things and not one was a defect inside the thing the agent built.

• A diagnostic route supplied a query scope the page route does not pass, so it reported a binding as resolving that the live page leaves unbound.
• A purge generation expired after a day, while a cache entry with the backstop turned off never expires, so a container could fall back onto a key another one still held.
• A replay claim was written before the purge it claimed, so a store that threw left the retry told the work was done.
• Animated blocks were correct on screen and empty to a screen reader, because the only text in the accessibility tree was hidden by the animation.

That is not carelessness and it will not go to zero. An agent has whole context on what it builds and partial context on what it calls. The tests it writes cover what it thought of, which by definition excludes what it did not.

So the target is not fewer findings. It is earlier ones. A finding caught after the push costs push, CI, review, fix, push, CI again. The same finding caught before the push costs one local test run. On a .NET repository where the test job is twenty-five minutes, that is the difference between an hour and a minute.

interaction-surface.sh prints the part the agent did not think of. For a diff it answers two questions: which unchanged files use a symbol the diff defines, and which unchanged files import a changed file. It finds candidates, it does not judge them. For each entry the author must produce one of two things before pushing: a test that crosses that boundary, or a sentence saying why that caller is unaffected. An entry with neither is where the critic finds its next defect.

./interaction-surface.sh origin/master      # a branch
./interaction-surface.sh --wip              # uncommitted work

I checked it against a case whose answer I already knew. The pull request that added per-collection settings changed src/related.ts and left src/screens/post-view.tsx alone. The script names that exact pair. The defect that escaped that pull request was in that exact pair, and it was fixed two pull requests later. One local run, before the push, would have put it in front of the author.

Two callers deserve a second look whatever the list says, because both produced real defects above while every test passed: a path that reads configuration the change also reads, and a path that runs when the change fails rather than when it succeeds.

What the sixth pull request did to that claim

The next change through was bigger than the five before it, and it broke the pattern. Six findings: three were interactions, a search control pointing at a route that cannot read a query, one of two code paths clamping a limit while the other did not, and a fallback that survived for the default configuration but not a replaced one. The other three were defects inside the new code, an unfiltered read, an unvalidated repeated query parameter, and a fixture linking to a page nothing served.

So the honest claim is most, not all, and it thins out as a change gets larger and carries more new surface of its own. A tool aimed at interactions cannot reach the other half, and nothing here should be sold as if it can.

The same run gave the tool its first real use and two corrections worth taking. The author ran it after pushing rather than before, so it was not a clean test of whether it reduces findings. It printed fifteen symbol groups over fifty files, of which about half were pure name collisions, a local called order in one module matching an unrelated local of the same name elsewhere. The fix was to match exported declarations only rather than exported or top level: a caller cannot depend on what it cannot reach, so a name nothing exports is not a boundary. On the same branch that took fifteen groups down to three, and two of the three were the exact surface the change was about.

The author's own verdict is the part worth keeping. Reading fifty entries to find three worth acting on still beat a review round trip, but the value was not the file list. It was the two questions at the end, which are prompts rather than search results, and which produced the two tests that were missing: a scope that could shadow a tenant's own field name, and a read whose failure must not throw because it would break a consumer's build. It also surfaced one thing that would otherwise have shipped, a second copy of a mapping that the change had just made redundant.

A list of files is cheap to generate and expensive to read. A short list of questions about the boundary is the opposite. Weight accordingly.

The second run, used properly, and the answer is no

The next change ran it before pushing rather than after, which is the test the first one could not be. It did not reduce review findings.

That change drew fourteen findings from elsewhere: three from the code scanner on regular expressions that stripped scripts, ten from an adversarial reviewer, and one from the review bot. The script found one thing, a latent name collision between two files in the same directory exporting the same two type names, which nothing imported together yet. It prompted five tests that are worth having. It did not find a single one of the fourteen.

Two of those fourteen were serious and neither was reachable from a file list. A block named for sticking never stuck, because every block is wrapped in a div exactly as tall as its contents and a sticky element cannot move outside its containing block; measured, its top went from 0 to minus 400 after a 400 pixel scroll, and no test or screenshot could see it because the look check captures at rest. And a test runner checked that something answered on a port rather than that the thing answering was the process it had just started, so a stale server would have passed the whole suite.

The author's own summary was that every bug that mattered came from somewhere else.

So, honestly:

• The import half earns its place. Which unchanged files import a file you changed was useful on both runs. It is a small list and every entry is a real dependency.
• The symbol half did not, until it was narrowed. Before matching exported declarations only it was about ninety five percent noise, local names colliding with unrelated local names. After, it went from eighteen groups to three on the same branch. Even then it produced no defect.
• The two questions did more work than the tool. One of them, the path that runs when the change fails, named exactly the category of the stale-server hole. The script did not find it. The adversarial reviewer did. A question that makes an author go and look beats a list that makes them go and read.

The honest conclusion is that this is a cheap pre-filter that buys a few tests and the occasional latent trap, and it is not a substitute for an adversarial pass. If it stays, keep the import half and the questions, and treat the symbol list as optional.

That is two changes of evidence, not twenty, and both were large. A smaller change with a narrow surface may well look different.

The third run, which pays for the other two

The next change was a narrow one, four screens that each handled the same optimistic concurrency response their own way. The symbol list printed nothing at all, both times it ran. The import list printed twenty four entries, eleven crossed by an existing or new test, twelve dismissed with a sentence in the pull request.

The twenty fourth was a real defect that would have shipped. A presets module appeared on the list only because it imports the site module. It turned out to hold a fourth answer to the same conflict response, writing the same document, which meant the change's own title was wrong and one screen would still have told a person to try again where the other three now merge silently. Nothing in eight hundred tests pointed at it. The list did.

Both of the two questions paid on that change too. The configuration one found that re-seeding after a save had dropped the header a deployment can be configured to require, so the second save from an open editor would have been refused. The failure-path one is the whole subject, and following every caller of the three old handlers into the one new handler is what surfaced the fourth.

So across three changes: the symbol list produced no defect on any of them, and was pure noise on two. The import list produced nothing, nothing, and then the one defect that mattered. The questions paid on all three.

The symbol list is now off by default, behind --symbols. Keeping it on meant paying to read fifty entries on every change to find the three that were real, and it never once held the answer.

Two other things that run tells you. That change also found two silent overwrites, both of which wrote an older document under a precondition the server accepted, so somebody's edit vanished with no error anywhere. Neither came from any reviewer; they came from unifying four code paths and seeing what they disagreed about. And the review bot returned nothing on it at all, the first time all day, on the change with the most dangerous bugs in it. A quiet reviewer is not evidence of a clean change.

The other half of this is that the operator is a failure source too, and a cheaper one to fix. Three traps from the same run, all mine rather than an agent's:

• Run the gates that consume a change, not only the gate that checks it. After bumping thirteen package versions I ran the version gate and shipped. The test that guards the template's pinned version failed in CI, and fixing that made the template package itself a changed version, which failed the same gate again. Two full test rounds for something one local run would have caught.
• A warning about the thing you just changed is the failure, not noise. That same run printed BarakoCMS.Testing 4.0.0 was not found, 4.3.0 was resolved instead on my own machine. I read it and judged it low risk. It was the CI failure, stated in advance.
• A wait loop that greps for a command string matches its own command line. Twelve background shells polled with pgrep -f 'dotnet test ...', each matched itself, and none could ever exit. They also kept each other alive. Watch a file the run writes, or exclude your own process id.

One honest tension with section 1, which says a review finding is fixed in the change or dropped. That run filed twelve follow-up issues. Most were genuinely out of scope, an API change in another repository or a product decision an agent should not make alone, and every actual review finding was fixed in its change. But twelve is close enough to a growing backlog that it is worth counting next time rather than trusting the distinction.

---

Previous: The adversarial critic | Home | Next: Cheap drafting works, and it stops short