Support concurrent mutation of disjoint entity sets (blocked: single-threaded work first)

[herve-quiroz]

Context, and why this is not ready to start

This records a design requirement discovered while planning parallelism for the sibling projects lockstep and nrg. It is deliberately blocked: the single-threaded optimisations (trancecode/vantage#15, #16, #17 and lockstep's in-flight-effect indexing) must land and be measured first, because they change where the performance ceiling actually is. Recent measurement found a single four-line pathfinding fix worth 176x to 374x, so parallelism is not the next lever.

The requirement

A future parallel simulation scheme would run disjoint sets of entities on separate goroutines within one world, synchronising at barriers. This requires the ECS to tolerate concurrent mutation of entity sets that provably do not overlap.

Today it does not. Per the design that nrg/ARCHITECTURE.md describes: typed Accessor[T] handles return interior pointers into per-type dense storage, and structural changes can reallocate that storage. A structural change on one goroutine can therefore invalidate a pointer held by another, even when the two goroutines are touching entirely different entities.

What the consumer needs

• Concurrent reads and writes to components of disjoint entity sets, safe without a global lock.
• Structural changes (create, destroy, add or remove component) that do not invalidate pointers held by other goroutines operating on unrelated entities.
• Determinism preserved. Both consuming games depend on bit-identical replay and savegames, so any parallel path must produce results identical to the sequential path, and merges must happen in a fixed order. This is a hard requirement, not a preference.

Possible directions, not prescriptive

• Stable storage that never reallocates under structural change (chunked or paged storage).
• Handle-based access instead of interior pointers on the concurrent path.
• An explicit partitioned view or borrow API that hands out disjoint slices of the world.

Acceptance

• A written design describing the chosen approach and its determinism argument, agreed before implementation.
• A race-detector test that mutates two disjoint entity sets from two goroutines and asserts results identical to running them sequentially.
• No regression on single-threaded performance, which remains the common case.

---

✨ Content generated by Claude AI.

[herve-quiroz]

Opened #15 with the design this issue's first acceptance criterion asks for. It is documentation
only — no code, nothing implemented.

Implementation stays blocked, as this issue says it should. I verified that trancecode/vantage#15,
#16 and #17 are all still open, so the single-threaded work that determines where the ceiling
actually sits has not landed or been measured. #15 therefore references this issue rather than
closing it; this issue stays open for the race-detector test and the no-regression work.

The proposed approach, briefly: structural quiescence. Inside a parallel region the world is
structurally frozen — no storage allocation, growth, compaction or reordering, no map writes, no
liveness changes. Only value writes into slots that already exist, each by the one goroutine that
owns that entity. Nothing moves, so no interior pointer can dangle; concurrent map reads with no
writer need no lock; disjoint-address writes commute; and structural changes are buffered per
goroutine and applied by one goroutine at the barrier in group-index order. The sequential path is
left byte-identical, which is what protects the third acceptance criterion.

The question that needs settling before anything is built. "Structural changes that do not
invalidate pointers held by other goroutines" reads two ways: visible at the barrier, or visible
immediately inside the region. The design takes the first — it matches the barrier-synchronised
model described here, and it is already the semantics the sequential API has during iteration. The
second reading means replacing the sparse-set storage model wholesale and paying for it
single-threaded.

If either lockstep or nrg needs one group's structural change to be observable by another group
before the barrier, the design is wrong and needs reworking. Four smaller open questions (identifier
block sizing, savegame impact of a sparser id space, the Bind API shape, and whether cross-group
reads are needed) are listed at the end of the document.

---

✨ Content generated by Claude AI.

[herve-quiroz]

Work detection selected this issue again for implementation, so a note on why no further code is coming and what unblocks it.

The design deliverable — acceptance criterion 1 — is already open as #15, awaiting agreement. Detection missed it because #15 deliberately says Related: #14 rather than fixes #14: the design does not close this issue, since criteria 2 (race-detector test) and 3 (no single-threaded regression) remain. The consequence is that this issue is re-selected every session while #15 sits unreviewed.

I have added input required to stop that loop. It is accurate rather than procedural: criterion 1 says the design must be agreed before implementation, and #15 asks a question only a human consumer can answer — whether one group's structural change must be observable by another group before the barrier, or whether at the barrier is enough. The whole design turns on that answer, so building the race-detector test now would risk testing semantics that get discarded.

Independently, trancecode/vantage#15, #16 and #17 are all still open, so the single-threaded work this issue is explicitly blocked behind has not landed or been measured.

To resume: answer the barrier-visibility question on #15 and merge it, then remove input required here.

---

✨ Content generated by Claude AI.