prompt: the phase-2 arms collide — the clipper is not in the search identifier - #193
Merged
Merged
Conversation
…dentifier Verified empirically against PyAutoFit main: no-clipper, ClipperNone and ClipperPriorBox all resolve to identifier 2bada4747f74bc46bf812605a762def9, so arms 1 and 2 of the validation campaign -- which differ in nothing else -- write to the SAME output directory. Stacked with the .completed short-circuit (is_complete -> result_via_completed_fit, which skips the search entirely), arm 2 can silently return arm 1's numbers and look like a clean run. That invalidates the campaign's central comparison while producing output that looks perfect. Credit: this was first identified by PyAutoMind#189, a parallel session's phase-1 ship PR that is still open. Re-verified here rather than relayed. Mitigation written into the prompt: unique `name` per arm (do not rely on the clipper to separate them), delete output/<name>/ between arms anyway, and assert the recorded step count equals n_steps -- a short-circuited run reports the PREVIOUS run's numbers rather than zeros, so a zero-check alone does not catch it. Also sharpens the clip-count validity check: an arm showing no clipping lines at all is the signature of this collision, since arm 1's summary carries no Clipper line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR
Jammy2211
pushed a commit
that referenced
this pull request
Aug 29, 2026
…#193); dashboard regenerated Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoFQxXV9W59icpEyiGL4eG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A campaign-invalidating trap that the phase-2 handover was missing. Urgent, because the campaign is about to be handed to a GPU session.
The trap
The clipper does not enter the search identifier. Verified empirically against PyAutoFit
main:All three resolve to the same output directory. Arms 1 and 2 of the validation campaign differ in nothing else, so they collide. Stacked with the
.completedshort-circuit (is_complete→result_via_completed_fit, which skips the search entirely), arm 2 can silently return arm 1's numbers and look like a clean run — invalidating the campaign's central comparison while producing output that looks perfect.This is not a bug in itself: it is what keeps existing runs' output directories stable across the phase-1 change. It is just precisely wrong for a campaign whose arms differ only in the clipper.
Mitigation written into the prompt
nameper arm — do not rely on the clipper to separate them, because it does not.output/<name>/between arms anyway.n_steps. A short-circuited run reports the previous run's numbers rather than zeros, so a zero-check alone does not catch this.Also sharpens the clip-count validity check: an arm showing no clipping lines at all is the signature of this collision, since arm 1's summary carries no
Clipperline.Credit and provenance
This was first identified by #189, a parallel session's phase-1 ship PR that is still open. I re-verified it against
mainrather than relaying the claim, and the identifier hash above is from that check.#189 overlaps my #190 — both wrote a phase-1 completion record, under different slugs (
autofit-prior-support-clipper.mdthere,prior-support-clipper.mdhere). I did not check for open PRs before starting, which is how the duplication happened. #189 is now behindmainand would conflict. It also carries content that has not landed anywhere — the LBFGSoptimizeprocess lesson, several harness traps, and two follow-ups (declaringLogGaussianPrior's(0, ∞)support on the prior itself, and whether the clipper belongs in the identifier). That needs a human decision, so I have left #189 alone.🤖 Generated with Claude Code
https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR
Generated by Claude Code