ios-agent-visual-loop

A small, fully reproducible demo of closing the visual feedback loop for an iOS coding agent: a SwiftUI fixture app, a deterministic frame-posing hook, and an independent pixel-level oracle. Companion repo to the write-up (in French): Useless Harness Experiment.

Why

On code that build and tests can judge, an agent's feedback loop already exists. On visual work it usually doesn't: the agent ships geometry it has never seen. This repo makes a UI animation verifiable in pixels, and shows what an independent oracle guarantees that "the code is correct" does not.

What's here

• Postcards/ is the fixture: a stories viewer (5 decks of image cards, horizontal deck-to-deck navigation), regenerated with XcodeGen from project.yml. No third-party dependencies.
• docs/cube-transition-spec.md is the executable spec of the target animation: a 3D cube turn between decks, every quantity numbered (projection, seam trajectory, letterbox profile, tolerances), frozen before any implementation run.
• harness/ is the verification loop: capture.sh poses any transition frame through a DEBUG-only --scrub launch hook and screenshots the simulator deterministically; assert_cube.py returns a JSON verdict per frame against the spec's model; selftest_cube.py renders negative controls from the committed baselines (a door swing, a mirrored face, a missing perspective, a plain slide must all be rejected) and needs no cube implementation to run. Details in harness/README.md.

The demo

The transition on main was implemented by a coding agent against the spec with the harness in the loop: it ran the oracle's self-test first, then iterated on capture and assert until every pose passed, without touching harness/. Its brief is committed as docs/cube-task-brief.md, and an independent evaluation of the delivered branch passes the oracle on all 8 checkpoints.

The "before" lives on illustration/doorswing: the same fixture with a broken geometry reproduced from a real-world miss (two faces hinged on opposite screen edges, no shared edge, no translation). The oracle rejects it on every transition pose from a single scrubbed frame (at p = 0.25, 8 of 11 checks fail) and passes it at rest.

Videos of both are embedded in the write-up.

Reproduce

harness/selftest_cube.py              # oracle self-test, no build needed
harness/capture.sh 1 0.5 frame.png    # pose deck 1 at mid-transition
harness/assert_cube.py --deck 1 --progress 0.5 frame.png

Requires Xcode with an iPhone 17 / iOS 26.5 simulator, and XcodeGen.

Branches

The whole experiment is kept auditable, not just its conclusion.

• main is the fixture plus the retained deliverable (the loop cell, merged as delivered).
• illustration/doorswing is the reproduced "before".
• demo/* are the four starting points handed to the agent (bare, noverify, blind, loop): same fixture, same task, different verification scaffolding. The diffs between their briefs are the experiment's design.
• vm/* are the four deliveries, exactly as the agent pushed them, one per cell. All four pass the oracle 8/8.
• eval/bare-hooked is the bare delivery with the scrub hook cherry-picked back on, the form the oracle evaluated.

License

MIT, see LICENSE.