implement loses two of its five beats on unattended runs (review before commit, bare sub-skill names) #1091
Replies: 1 comment
|
I can confirm points 1 and 2 with numbers. afk.sh (#1136) runs Until
On point 3 I went a different way than the staging step. The ticket's acceptance criteria already are a written confirmation that someone read before the run, so the prompt tells That part isn't proven yet. The harness checks what the prompt says, not what a real session does with it. After the next real run I'll count the The real fix is still the one you describe: swap the last two lines and use the namespaced names. |
Uh oh!
There was an error while loading. Please reload this page.
*Written with Claude's help otherwise these would be 5 times longer with me trying to get my meandering thoughts across.
I've been running
/mattpocock-skills:implementon small batches of tickets overnight in Claude Code (plugin install, 1.2.3). On more than one run, two of its five beats quietly didn't happen, and I only found out by reading the transcript the next morning. Each cause traces back to the skill body, which is still the same on main: (the relevant three lines)The review runs before the commit.
code-reviewdiffs HEAD against a fixed point, so reviewing before committing hands it an empty diff. Its own guard would catch that, but only when a fixed point is supplied, and implement never passes one. Swapping the last two lines and saying "pass the fixed point explicitly" fixes it./tddand/code-revieware bare names. The plugin namespaces them asmattpocock-skills:tddandmattpocock-skills:code-review, and bare/code-reviewresolves to Claude Code's built-in review instead. On one run the agent ran the built-in and skipped tdd entirely, with nothing in the output saying so. It looks like Standardize cross-skill invocation on "call the Skill tool" phrasing #878's "Call the Skill tool with" pass didn't reachimplement. A line like "if a beat can't run, say so rather than substituting or skipping" would also have made this visible."Pre-agreed seams" has no unattended path. A design question rather than a bug.
tddsays no test is written at an unconfirmed seam, and unattended there's no one to confirm with, so skipping tests is arguablytddfollowing its own rule. My tickets don't name seams, so I've tried a staging step: before the run, a normal attended session reads the tickets and the code and drafts the seams for each ticket. I read that list, and it goes into the run prompt as the written confirmation. I've only done this on one batch so far, so it's a suggestion rather than a proven pattern. My local copy also stops at the start if a ticket has no entry, instead of guessing. I'd be curious whether you'd want something like that inimplement, or whether unattended runs are whatimplement-specis meant to cover.For now I've got a local copy that does all this, but I'd rather track upstream than keep a fork.
All reactions