Round 38 milestone: owner_epoch — ownership as a durable fence (#432 + #413/#379/#384)
CORRECTS A FALSE SAFETY CLAIM. Three pages asserted that state_version
"fences stale writes, so a mis-fired handoff or a zombie owner can never
clobber a newer record". It never did: state_version is contention control,
and every caller answered a CAS loss by rebasing, so a stale writer rebased
exactly as a legitimate one did.
- Persistence-and-Durability: "The state_version guard" reworked into "Two
guards" — state_version (contention) vs owner_epoch (ownership, at the
SINK). Documents the rollback primitive it closed (a stale shard 60s-old
logout snapshot force-writing over the live owner, repeatable), why the
fence had to be a SECOND column (a rebase moves state_version and cannot
reach a separate conjunct), why the mint is atomic (a read-then-bump would
look fixed), the three-outcome SaveResult, terminal epoch loss on both save
paths, the max(directory,row) floor, fail-closed login, the checkpoint tier
ownership guard (it had NO CAS and was a full bypass one rung up; detects a
double-own ~6x sooner), the (owner_epoch, state_version) load ordering, and
the honest residual (an epoch on row X cannot fence a write to row Y).
- Distributed-Systems-Model + Overview: same correction; adds a "do not confuse
the two epochs" invariant (directory epoch fences ROUTING and is evictable;
owner_epoch fences WRITES and is durable).
- Cross-Shard-Handoff: handoff mints the epoch AFTER destination resolution
(minting first evicted legitimate players mid-move); an unreadable store now
fails the handoff closed; concurrent claims derive distinct epochs hence
distinct handoff tokens, closing the shared-pending race structurally.
- Zone-Runtime: #413 extends the resolve-then-deliver claim to login attach and
Prepare (incl. why neither may reuse claimTransferTarget, and Prepare needing
a release on post failure); #379 refuses a reconnect racing an in-flight
transfer in TWO layers, since a probe reproduced a dupe the resolve-time
check alone cannot see.
- #384: documents why the drain-reservation margin is not bound by step 1
duration (the hold is rebased per accumulate and anchored to the redirect it
covers) and the WARN added to falsify it; corrects the Round 31 "stamped once
and never refreshed" wording on three pages.
Round 36 milestone: instanced zones (#72)
New deep-dive page Instanced-Zones, plus corrections to two pages that
asserted "instancing is deferred / not implemented".
- NEW Instanced-Zones: template-vs-id identity and the ownsZoneRef locality
chokepoint (with the build-failing AST lint); why the id is <template>#<128-bit
random> (# outside the ref charset, unguessable so it is not a farming oracle);
unleased by construction; async 3-hop mint + quiescence reaper; the
instanceable content opt-in; the isolation exclusions (persistent resets fail
closed vs a durable item dupe, per-mint RNG salting, withheld director
schedule events, refused signal_region/world, drain/reload/off-box-ingress);
THE ANCHOR (template projection was killed because placement is the gate ROUTING
key); exit paths; and the security posture.
- Distributed-Systems-Model + Running-at-Scale (x2): instancing now exists but is
ISOLATION, not transparent load-sharding — the public hot-zone ceiling stands.
- Zone-Runtime: quiescence gains the third `incoming` counter for in-flight
intra-shard transfers, claimed under the same mutex teardown checks.
- Cross-Shard-Handoff + RPC-and-Protobuf: signed-snapshot optionals now written
UNCONDITIONALLY — a second append-if-non-empty optional made
digest(tier="",account=X) == digest(tier=X,account="") (length prefixes solve
boundaries, not presence); instances rejected at every off-box ingress.
- Persistence: an instance is never a durable location; the anchor + COALESCE.
- Pack-Entity-Reference: zone `instanceable` flag, start_room respawn semantics.
- Pack-Lua-Scripting: mud.zone(), mud.send_to_instance (self-only).
- Orchestration: instances withheld from reserved schedule fan-out.
- Sidebar + Engine-Developer-Reference index entries.
Round 32 milestone: comms & JetStream durability residuals
- Scoped-Event-Bus: MaxAckPending is now a per-consumer option (default 1,
double-clamped vs NATS 0==unlimited) rather than a hardcoded transport
constant; both ordered consumers keep the serializing posture, and >1 is the
lever a future seen-set reorder-tolerant consumer would pull (#312).
- Running-at-Scale + Sysadmin-Reference: durable_parked_total is now counted
off the broker MAX_DELIVERIES advisory via a per-stream queue group (once
cluster-wide), so it also catches an ack-wait-expiry or across-restart park,
not just an in-process final delivery — an alert signal, not an exact ledger
(the advisory is ephemeral) (#311).
- Distributed-Systems-Model: placement is an existence/routing oracle, not a
liveness one; a "currently connected?" decision (e.g. the new-mail ping)
reads the presence roster and only to gate, never to route (#325).
- Content-Loading-and-Hot-Reload: the comms hear-set republish retry and the
zone-shape reconcile retry run on separate goroutine budgets at parity, so a
reconcile storm cannot starve the security-relevant republish class (#345).
(#371 async-who prompt ordering is an internal fix with no documented surface.)
Round 31 milestone: handoff phase-2 auth, checkpoint tie-break, drain-reservation TTL
- Cross-Shard-Handoff + RPC-and-Protobuf: document that Handoff.Commit/Abort
are now Ed25519-signed under the cluster keypair with the digest bound to the
destination shard id (#314) — the deterministic handoff_token over public
inputs was forgeable, and destination binding stops a captured signed Abort
from being replayed against a second destination in a split-brain race.
- Persistence-and-Durability + Distributed-Systems-Model: the load freshness
check breaks a state_version tie toward the Redis checkpoint (#322); a
checkpoint is dumped at the pre-CAS version so it ties the row while carrying
newer content — the strict > left the tier inert, so the tie-break is what
makes the crash window the ~10s cadence and not the ~60s flush.
- Cross-Shard-Handoff + Running-at-Scale + Distributed-Systems-Model: the
drain-target reservation TTL now outlasts the whole drain (deadline + one
presence-reflect window) so a slow-but-alive drain keeps its hold while
players migrate; only a crashed drainer lingers the full TTL (#334).
Round 30 milestone: reconnect/placement/drain correctness (#321/#324/#329/#331/#336) + RID re-home (#373)
Round did not self-document. Closed:
- Persistence-and-Durability: a live reconnect routes by an in-memory residency
index BEFORE the lagging durable zone_ref (prevents a double-owned character
when a link-dead resume beats the async flush of an intra-shard walk) (#321);
the placement writer gets the saver's shutdown-barrier treatment so a logout
tombstone isn't lost on graceful shutdown (else the tell/mail oracle reports a
player on a dead shard) (#331)
- Distributed-Systems-Model: the placement tombstone now carries a per-session
nonce, fencing a same-shard/same-epoch relog that shard+epoch alone couldn't
distinguish (#329)
- Edge-and-Protocol: a fresh login bounced by a draining shard (Unavailable) is
re-resolved + re-dialed to the peer instead of dropped; token-bearing re-dials
never retried (extends the zero-drop drain to ARRIVING players) (#324)
- Cross-Shard-Handoff: a gate-wedged player is skipped by the drain and reclaimed
(client-fault straggler), not miscounted zero-drop Redirected (#336)
- Entity-Component-Model: an intra-shard transfer re-homes the entity + carried
subtree to fresh per-zone RIDs (the overworld 'map on some rooms' bug) (#373)
Round 26: the placement record becomes the reconnect-routing spine
Track-6 residuals. Round self-documented only #276 (catch-up metrics); the
rest of the engine/ops surface was stale.
- Distributed-Systems-Model: NEW 'The placement record' section — zone is the
routing key (ShardForZone resolves the current owner, so an offline rebalance
is transparent), epoch is the fence AND the tell/mail existence key, shard
routes nothing. Every residency registers a placement (incl. the intra-shard
zone walk); registerPlacement accepts an equal epoch where the handoff CAS
demands a strictly greater one. Clean logout writes a fenced compare-and-delete
tombstone, not a delete (#320 slice 2, #70). Also: lease renewal restarts on
re-adoption, closing an A->B->A split-brain (#288).
- Persistence-and-Durability: close the shard-not-zone 'Known gap'; add the
saver shutdown sentinel barrier (#282).
- Edge-and-Protocol: world-side Send watchdog — keepalive can't see a gate whose
transport acks PINGs while its application stopped reading (#274).
- Orchestration-and-Directors: a runtime-adopted zone's scope replica is seeded
before the zone is exposed (a full-map replace would clobber newer state) (#280).
- Cross-Shard-Handoff / Running-at-Scale / DistSys: drain-target reservations now
carry per-field, server-timed expiry (#284).
- Running-at-Scale: drop the self-contradicting 'deliver_lag_ms silent' gap; the
gate row now describes zone-keyed reconnect routing.
Scaling docs: reflect shipped load-aware placement rebalancer
Track 6 drain/scale landed: occupancy-weighted rebalancing that executes
(director issues rebalance-drain directives with cooldown + locality colocation)
and director-owned, reservation-based drain-target selection. Updates the
now-stale 'advisory-only / naive drain-select' language on Running-at-Scale,
Distributed-Systems-Model, Cross-Shard-Handoff, and Orchestration-and-Directors.
Engine Dev: directors, scoped event bus, handoff, distsys model
- Orchestration-and-Directors: director-as-actor, per-scope leader election,
scope hierarchy, worked boss loop; unauthored-director-script flag
- Scoped-Event-Bus: two lanes, write-up/read-down, apply-once watermark,
the no-cross-scope-budget nuance
- Cross-Shard-Handoff: two-phase Prepare/Commit, signed snapshot + tier carry,
graceful drain, gate redirect/replay
- Distributed-Systems-Model: single-writer spine, leases, placement, the
one-zone-one-core ceiling + honest scaling gaps
Scaffold wiki + author About and Getting Started
- 48-page skeleton across all audiences (flat files + grouped _Sidebar)
- Every page carries an Audience + Status banner (Ready/Draft/Placeholder)
- About: Overview, History of MUDs (Ready)
- Getting Started: Getting-Started, Running-Locally, First-Admin-Setup,
Content-Packs-Intro (Ready) — dev bypass, local GitHub OAuth, bootstrap
admin, promote/demote, content-pack model, all grounded in code
- SRD5/WoW/D6 pack pages are honest placeholders (packs not yet built)