Skip to content

test(cliffs): raising n - WEST replicates, "west specifically" does NOT (#84) - #153

Merged
wormeyman merged 1 commit into
mainfrom
cliffs-west-out-of-sample
Aug 5, 2026
Merged

test(cliffs): raising n - WEST replicates, "west specifically" does NOT (#84)#153
wormeyman merged 1 commit into
mainfrom
cliffs-west-out-of-sample

Conversation

@wormeyman

Copy link
Copy Markdown
Owner

The check six mechanism hunts were spent without. #150's z = 3.01 edge split and #151's four sweep-order arms all reuse the same 14 regions, so every one of them was a re-slice of a single sample - the shape this repo has been burned by before.

New capture: oracle-vulcanus-cliff-entities-west-oos, eight fresh Vulcanus regions with the same paired ON / ALL-resources-OFF ore lever, disjoint from all 15 already in use and spread away from spawn and from each other.

unexplained on border W N E S z(W) z(N) z(E)
in sample (14 regions) 25 19 9 5 3 2 2.60 0.41 −1.21
out of sample (8 new) 34 23 10 9 4 7 2.29 1.79 −1.04

What replicates

West is the most enriched edge again (z = 2.29), and east is the depleted edge in both samples. The concentration is real - it is not an artifact of re-slicing one sample.

What does not

"West specifically." In sample, north sat at its base rate (z = 0.41) and west led it by more than 2 sigma. Out of sample north is 1.79, trailing west by half a sigma. The claim that survives both samples is weaker and differently shaped:

the low-coordinate edges - west and north - are enriched, and east is depleted.

That widens the target rather than narrowing it, and it matters because "west" and "west + north" point at different mechanisms. Note #151 already refuted, by permutation, the one rule whose asymmetry is literally west-then-north: the repair sweep's L, T, R, B clear order.

The cost of not doing this earlier

#150's headline was a 4-bin split on 17 cells from one sample. Its Bonferroni note was the right caution about the wrong risk: the multiple-comparison correction survived, and it was the sample that did not. Raise n before slicing again.

Contents

  • the vulcanus-cliff-entities-west-oos capture target
  • the fixture, and its PROVENANCE.json entry (2.1.12; pnpm refs:sync --check reported in sync at capture time)
  • test/cliffWestOutOfSample.spec.ts, which asserts both halves - what replicated and what did not

Verification

pnpm run verify green - 1588 app (up 4) + 13 worker + 3 container tests, 0 warnings, exit 0. Measurement only; shipping accuracy unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HKjzpWJw1HbnigPP9udqN2

…OT (#84)

The check six mechanism hunts were spent without. #150's z = 3.01 edge split and
#151's four sweep-order arms all reuse the SAME 14 regions, so every one was a
re-slice of one sample - the shape this repo has been burned by before.

NEW CAPTURE: oracle-vulcanus-cliff-entities-west-oos, eight fresh Vulcanus
regions with the same paired ON / ALL-resources-OFF lever, disjoint from all 15
already in use and spread away from spawn and from each other.

                          unexpl  border   W  N  E  S   z(W)  z(N)   z(E)
  in sample (14 regions)      25      19   9  5  3  2   2.60  0.41  -1.21
  out of sample (8 new)       34      23  10  9  4  7   2.29  1.79  -1.04

WHAT REPLICATES: west is the most enriched edge again (z = 2.29) and east is the
depleted edge in both samples. The concentration is real and is not an artifact
of re-slicing one sample.

WHAT DOES NOT: "west specifically". In sample north sat at its base rate
(z = 0.41) and west led it by more than 2 sigma; out of sample north is 1.79 and
trails west by half a sigma. The claim that survives both samples is weaker and
differently shaped - the LOW-COORDINATE edges, west AND north, are enriched and
east is depleted.

That widens the target rather than narrowing it, and it matters because "west"
and "west + north" point at different mechanisms. #151 already refuted by
permutation the one rule whose asymmetry is literally west-then-north: the repair
sweep's L, T, R, B clear order.

Cost of not doing this earlier: #150's headline was a 4-bin split on 17 cells
from one sample, and its Bonferroni note was the right caution about the wrong
risk - the multiple-comparison correction survived and it was the SAMPLE that did
not. Raise n before slicing again.

Adds the capture target, the fixture, its PROVENANCE entry (2.1.12, refs:sync
--check reported in sync at capture time), and the spec.

Verified: pnpm run verify green - 1588 app (up 4) + 13 worker + 3 container,
0 warnings, exit 0. Measurement only; shipping accuracy unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HKjzpWJw1HbnigPP9udqN2
@wormeyman
wormeyman merged commit 81dde28 into main Aug 5, 2026
6 checks passed
@wormeyman
wormeyman deleted the cliffs-west-out-of-sample branch August 5, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant