Retire the swarm.points writable stack: read-only snapshot, coords is the write path (#379 item 1)#382
Conversation
Adversarial review — round 1Findings from an independent reviewer running live repros against the stack build:
Fixes follow on this branch. Underworld development team with AI support from Claude Code |
…deform re-bins swarms (#382 review) Adversarial review round 1 findings, all addressed: 1. In-repo writers of swarm.data (test_0006 particle jiggle; three Coriolis streakline examples) migrated to swarm._particle_coordinates.data — they broke against the read-only snapshot with numpy's bare error. The snapshot itself is now a thin view whose write attempts raise WITH the guidance (coords / _particle_coordinates.data / migration_control), since item assignment never reaches the property setter's message. 2. The persistent-variable transfer passes model-unit coordinates (_particle_coordinates.data) instead of swarm.coords: under active units, coords returns a pint Quantity that evaluate cannot consume, and the blanket except silently skipped the transfer. (The old swarm.points path was also broken here — it transferred corrupt values via a pre-existing unit-label bug.) 3. Retiring the migration-on-read trigger left the mesh->swarm change channel with zero consumers: a deformed mesh stranded particles permanently. Solve-entry sync (_sync_before_assembly) now compares the recorded mesh version and marks _needs_migration, slotting into the existing rank-agreed machinery. 4. Model summary uses swarm.local_size instead of len(swarm.data) (which now warned and built a full physical-coordinate copy per call). New tests: snapshot writes carry the guidance message; mesh-version change marks the swarm for deferred migration at solve entry. Underworld development team with AI support from Claude Code
1f0fb52 to
559ecba
Compare
Round 1 — response (fixed in 559ecba; branch rebased onto the updated #381)All four findings addressed:
The pre-existing 1000× unit-label bug in the physical-coordinate scaling (meter magnitudes labelled kilometres) is real and untouched here — it predates this PR and deserves its own issue rather than a rider on a retirement PR. Quick gate re-run in progress; Underworld development team with AI support from Claude Code |
…eview) Round-1 review: np.add(x, 1, out=x) bypassed the guard that x += 1 enforces, re-arming the per-write hazard the flag exists to prevent (its only in-tree user was the legacy swarm.points wrapper, whose callback ran collective migrate() — removed by #382, but the flag's contract must hold regardless). The hook now raises the same RuntimeError for any out= target carrying the flag, before any computation. Regression test added. Underworld development team with AI support from Claude Code
1f42128 to
d2cf355
Compare
…eview) Round-1 review: np.add(x, 1, out=x) bypassed the guard that x += 1 enforces, re-arming the per-write hazard the flag exists to prevent (its only in-tree user was the legacy swarm.points wrapper, whose callback ran collective migrate() — removed by #382, but the flag's contract must hold regardless). The hook now raises the same RuntimeError for any out= target carrying the flag, before any computation. Regression test added. Underworld development team with AI support from Claude Code
…deform re-bins swarms (#382 review) Adversarial review round 1 findings, all addressed: 1. In-repo writers of swarm.data (test_0006 particle jiggle; three Coriolis streakline examples) migrated to swarm._particle_coordinates.data — they broke against the read-only snapshot with numpy's bare error. The snapshot itself is now a thin view whose write attempts raise WITH the guidance (coords / _particle_coordinates.data / migration_control), since item assignment never reaches the property setter's message. 2. The persistent-variable transfer passes model-unit coordinates (_particle_coordinates.data) instead of swarm.coords: under active units, coords returns a pint Quantity that evaluate cannot consume, and the blanket except silently skipped the transfer. (The old swarm.points path was also broken here — it transferred corrupt values via a pre-existing unit-label bug.) 3. Retiring the migration-on-read trigger left the mesh->swarm change channel with zero consumers: a deformed mesh stranded particles permanently. Solve-entry sync (_sync_before_assembly) now compares the recorded mesh version and marks _needs_migration, slotting into the existing rank-agreed machinery. 4. Model summary uses swarm.local_size instead of len(swarm.data) (which now warned and built a full physical-coordinate copy per call). New tests: snapshot writes carry the guidance message; mesh-version change marks the swarm for deferred migration at solve entry. Underworld development team with AI support from Claude Code
559ecba to
73523d4
Compare
…deform re-bins swarms (#382 review) Adversarial review round 1 findings, all addressed: 1. In-repo writers of swarm.data (test_0006 particle jiggle; three Coriolis streakline examples) migrated to swarm._particle_coordinates.data — they broke against the read-only snapshot with numpy's bare error. The snapshot itself is now a thin view whose write attempts raise WITH the guidance (coords / _particle_coordinates.data / migration_control), since item assignment never reaches the property setter's message. 2. The persistent-variable transfer passes model-unit coordinates (_particle_coordinates.data) instead of swarm.coords: under active units, coords returns a pint Quantity that evaluate cannot consume, and the blanket except silently skipped the transfer. (The old swarm.points path was also broken here — it transferred corrupt values via a pre-existing unit-label bug.) 3. Retiring the migration-on-read trigger left the mesh->swarm change channel with zero consumers: a deformed mesh stranded particles permanently. Solve-entry sync (_sync_before_assembly) now compares the recorded mesh version and marks _needs_migration, slotting into the existing rank-agreed machinery. 4. Model summary uses swarm.local_size instead of len(swarm.data) (which now warned and built a full physical-coordinate copy per call). New tests: snapshot writes carry the guidance message; mesh-version change marks the swarm for deferred migration at solve entry. Underworld development team with AI support from Claude Code
…ds residual mutation routes (#382 review round 2) Round 2 verified all round-1 fixes closed and found one gap: the solve-entry swarm re-bin fires off _mesh_version, but _deform_mesh never bumped it — only the mesh.X.coords callback path did. So mesh.deform(), the sanctioned deform for variable-carrying meshes, still left registered swarms stranded. The version now bumps inside _deform_mesh (the callback path bumps again afterwards — harmless, the consumer checks inequality). Regression test added. Also recorded on the snapshot docstring: non-setitem mutation routes surface numpy's read-only error rather than the guidance (still refused), and re-enabling the base's writeable flag reaches only the detached copy, never the swarm. Underworld development team with AI support from Claude Code
73523d4 to
05ab799
Compare
…eview) Round-1 review: np.add(x, 1, out=x) bypassed the guard that x += 1 enforces, re-arming the per-write hazard the flag exists to prevent (its only in-tree user was the legacy swarm.points wrapper, whose callback ran collective migrate() — removed by #382, but the flag's contract must hold regardless). The hook now raises the same RuntimeError for any out= target carrying the flag, before any computation. Regression test added. Underworld development team with AI support from Claude Code
Round 2 — verification and one gap (fixed in 05ab799)Round 2 confirmed all round-1 fixes closed (snapshot mutation routes probed exhaustively — Two notes recorded on the snapshot docstring: under active units the Underworld development team with AI support from Claude Code |
…ot, coords is the write path (#379 item 1) swarm.points kept a private writable wrapper whose per-write callback ran collective particle migration — ranks writing unevenly deadlock in parallel — and raised a bare rank-local RuntimeError on size mismatch. Reading it could also trigger a collective (forced migration after mesh changes), so a read on some ranks only could hang. The masked-write idiom migration_control()'s own docstring advertised raised through this same wrapper. This is the mesh.points treatment (BF-18): the deprecated read returns a detached read-only snapshot with no side effects; the setter refuses loudly, pointing at swarm.coords (physical units) and swarm._particle_coordinates.data (model units, masked writes supported under migration_control()). The migration_control() docstring example now shows the interface that actually works, and the one live caller (model.py persistent-variable transfer) uses swarm.coords, matching its mesh branch. Quick gate (level_1 and tier_a): 410 passed. Swarm write/timestep MPI smoke at np=2: passed. Underworld development team with AI support from Claude Code
…deform re-bins swarms (#382 review) Adversarial review round 1 findings, all addressed: 1. In-repo writers of swarm.data (test_0006 particle jiggle; three Coriolis streakline examples) migrated to swarm._particle_coordinates.data — they broke against the read-only snapshot with numpy's bare error. The snapshot itself is now a thin view whose write attempts raise WITH the guidance (coords / _particle_coordinates.data / migration_control), since item assignment never reaches the property setter's message. 2. The persistent-variable transfer passes model-unit coordinates (_particle_coordinates.data) instead of swarm.coords: under active units, coords returns a pint Quantity that evaluate cannot consume, and the blanket except silently skipped the transfer. (The old swarm.points path was also broken here — it transferred corrupt values via a pre-existing unit-label bug.) 3. Retiring the migration-on-read trigger left the mesh->swarm change channel with zero consumers: a deformed mesh stranded particles permanently. Solve-entry sync (_sync_before_assembly) now compares the recorded mesh version and marks _needs_migration, slotting into the existing rank-agreed machinery. 4. Model summary uses swarm.local_size instead of len(swarm.data) (which now warned and built a full physical-coordinate copy per call). New tests: snapshot writes carry the guidance message; mesh-version change marks the swarm for deferred migration at solve entry. Underworld development team with AI support from Claude Code
…ds residual mutation routes (#382 review round 2) Round 2 verified all round-1 fixes closed and found one gap: the solve-entry swarm re-bin fires off _mesh_version, but _deform_mesh never bumped it — only the mesh.X.coords callback path did. So mesh.deform(), the sanctioned deform for variable-carrying meshes, still left registered swarms stranded. The version now bumps inside _deform_mesh (the callback path bumps again afterwards — harmless, the consumer checks inequality). Regression test added. Also recorded on the snapshot docstring: non-setitem mutation routes surface numpy's read-only error rather than the guidance (still refused), and re-enabling the base's writeable flag reaches only the detached copy, never the swarm. Underworld development team with AI support from Claude Code
05ab799 to
e02a828
Compare
…mposed #382 bumps _mesh_version in _deform_mesh too The #382 round-2 fix adds a version bump inside _deform_mesh (so mesh.deform notifies swarms); the coords-callback path bumps again. The consumer contract is inequality — the regression tests now assert the same, instead of pinning per-write step counts that broke when the sibling branches composed on development. Underworld development team with AI support from Claude Code
…eview) Round-1 review: np.add(x, 1, out=x) bypassed the guard that x += 1 enforces, re-arming the per-write hazard the flag exists to prevent (its only in-tree user was the legacy swarm.points wrapper, whose callback ran collective migrate() — removed by #382, but the flag's contract must hold regardless). The hook now raises the same RuntimeError for any out= target carrying the flag, before any computation. Regression test added. Underworld development team with AI support from Claude Code
Second of the #379 series, stacked on #381 (base branch is
bugfix/canonical-callback-guard; retarget todevelopmentonce #381 merges). Draft until then.What this does
swarm.pointswas meant to mirrormesh.points, but the mesh side already had its dangerous half removed in the 2026-07 audit (BF-18: setter raises, pointing atmesh.deform()). The swarm side never got that treatment — it kept an entire private writable stack: a cached callback wrapper, a per-write callback that ran collective particle migration (ranks writing unevenly deadlock in parallel), a bare rank-localRuntimeErroron size mismatch, and a read path that could itself trigger a collective (forced migration after mesh changes), so a read performed on some ranks only could hang. The masked-write example inmigration_control()'s own docstring raised through this same wrapper.This PR completes the symmetry:
swarm.points(still deprecated) returns a detached, read-only snapshot in physical units — no cache, no callback, no migration side effect.AttributeErrorwith the working alternatives spelled out:swarm.coords = valuesfor whole-array writes in physical units, orswarm._particle_coordinates.data[...]for model-unit and masked writes (which defer migration correctly undermigration_control()— the new test runs that exact idiom).migration_control()docstring example now shows the interface that actually works.model.pypersistent-variable transfer) now usesswarm.coords, matching its mesh branch.Nothing that worked is lost: partial writes through
swarm.pointsalready raised before this change; whole-array writes haveswarm.coords.Verification
tests/test_0060_swarm_points_retired.py(level_1 / tier_a, written first and shown to fail): read-only snapshot semantics, loud setter refusal, coordinate state untouched by a refused write, and the supportedcoords/ maskedmigration_control()paths doing what the old docstring promised.level_1 and tier_a: 410 passed, 1 pre-existing xfail.Underworld development team with AI support from Claude Code