Skip to content

History / Cross Shard Handoff

Revisions

  • Round 44: player aliases, named exits + traverse hook, overworld mob markers (#353/#370/#363) Three player-facing features; each lands in its existing home (plus the traverse hook added to the room-trigger catalog). - #353 player aliases: Builder-Commands (new "aliases expand at the split step" subsection + the alias/unalias command entries) + Player-Reference + the handoff snapshot field list (Cross-Shard-Handoff). Expansion runs at the split step BEFORE verb resolution, so an alias can target any verb and grants no privilege (MinRank runs on the resolved verb); cycle/depth-bounded and re-capped to the ingress byte cap. Distinguished from an engine command-alias (n->north). Durable + rides a signature-bound handoff field, re-sanitized on arrival. - #370 named exits + traverse hook: Builder-Commands (named-exit fall-through, consulted last, exits map only never entrances) + Player-Reference (any exit keyword is typeable) + Pack-Lua-Hooks (the cancellable traverse room trigger: block/return-false/redirect, FAILS OPEN so it is not a hard security boundary, redirect is budget-bounded and can name only an exits key so it cannot reach an instance entrance — the depth-0-typed-move invariant) + the demo Warded Sanctum example. - #363 overworld nearby-mob markers: Pack-Lua-Scripting (has_visible_creature() presence-only disclosure primitive: display-render only, both rooms open_sight, canSee-filtered, mobs-only, never re-opening the foreign-room anti-scry) + Demo-Pack (the minimap's ! marker + the open_sight flag on plains rooms; mutual opt-in).

    @yfronto yfronto committed Jul 24, 2026
  • Observability 3: distributed tracing (gomud #464-#470) The metric->trace->log pivot is real. Fills out the Observability page Traces section (previously "none yet") and corrects the now-stale "no tracing" claims in Running-at-Scale. - Observability (Traces rewrite + intro + diagram): the tracing foundation (TracerProvider on the same OTLP env gate, W3C propagator set unconditionally, head sampling with a 100% carve-out); what is traced — handoff/AdoptZone at 100% (root spans, per-hop children, error-distinguishing failures), session- attach (bounded to the handshake; the Play stream is a context anchor not a span), and the comms bus / zone mailbox (traceparent in the ENVELOPE not NATS headers so MemBus works too; span LINK not parent-child because JetStream is at-least-once; queue_wait_ms at dequeue); the deliberate no-per-command-span decision; the metric->trace exemplar pivot on busLag (and why tick_lag_ms has none); the trace->log traceHandler; and the cardinality-is-a-security-boundary discipline (metricZone template-only, no per-player subjects, gate host not host:port, baggage-free producer links). Documents the dev-vs-staging split: the compose overlay renders the full pivot via otel-lgtm bundled Tempo, but the reference k8s deployment still defers the Tempo backend. - Running-at-Scale: corrected "traces not wired yet"; reframed the occupancy template-label as the #470 security boundary (regression-pinned). - Cross-Shard-Handoff: noted this is the one path traced at 100%.

    @yfronto yfronto committed Jul 21, 2026
  • Round 38 milestone: owner_epoch — ownership as a durable fence (#432 + #413/#379/#384) CORRECTS A FALSE SAFETY CLAIM. Three pages asserted that state_version "fences stale writes, so a mis-fired handoff or a zombie owner can never clobber a newer record". It never did: state_version is contention control, and every caller answered a CAS loss by rebasing, so a stale writer rebased exactly as a legitimate one did. - Persistence-and-Durability: "The state_version guard" reworked into "Two guards" — state_version (contention) vs owner_epoch (ownership, at the SINK). Documents the rollback primitive it closed (a stale shard 60s-old logout snapshot force-writing over the live owner, repeatable), why the fence had to be a SECOND column (a rebase moves state_version and cannot reach a separate conjunct), why the mint is atomic (a read-then-bump would look fixed), the three-outcome SaveResult, terminal epoch loss on both save paths, the max(directory,row) floor, fail-closed login, the checkpoint tier ownership guard (it had NO CAS and was a full bypass one rung up; detects a double-own ~6x sooner), the (owner_epoch, state_version) load ordering, and the honest residual (an epoch on row X cannot fence a write to row Y). - Distributed-Systems-Model + Overview: same correction; adds a "do not confuse the two epochs" invariant (directory epoch fences ROUTING and is evictable; owner_epoch fences WRITES and is durable). - Cross-Shard-Handoff: handoff mints the epoch AFTER destination resolution (minting first evicted legitimate players mid-move); an unreadable store now fails the handoff closed; concurrent claims derive distinct epochs hence distinct handoff tokens, closing the shared-pending race structurally. - Zone-Runtime: #413 extends the resolve-then-deliver claim to login attach and Prepare (incl. why neither may reuse claimTransferTarget, and Prepare needing a release on post failure); #379 refuses a reconnect racing an in-flight transfer in TWO layers, since a probe reproduced a dupe the resolve-time check alone cannot see. - #384: documents why the drain-reservation margin is not bound by step 1 duration (the hold is rebased per accumulate and anchored to the redirect it covers) and the WARN added to falsify it; corrects the Round 31 "stamped once and never refreshed" wording on three pages.

    @yfronto yfronto committed Jul 19, 2026
  • Round 36 milestone: instanced zones (#72) New deep-dive page Instanced-Zones, plus corrections to two pages that asserted "instancing is deferred / not implemented". - NEW Instanced-Zones: template-vs-id identity and the ownsZoneRef locality chokepoint (with the build-failing AST lint); why the id is <template>#<128-bit random> (# outside the ref charset, unguessable so it is not a farming oracle); unleased by construction; async 3-hop mint + quiescence reaper; the instanceable content opt-in; the isolation exclusions (persistent resets fail closed vs a durable item dupe, per-mint RNG salting, withheld director schedule events, refused signal_region/world, drain/reload/off-box-ingress); THE ANCHOR (template projection was killed because placement is the gate ROUTING key); exit paths; and the security posture. - Distributed-Systems-Model + Running-at-Scale (x2): instancing now exists but is ISOLATION, not transparent load-sharding — the public hot-zone ceiling stands. - Zone-Runtime: quiescence gains the third `incoming` counter for in-flight intra-shard transfers, claimed under the same mutex teardown checks. - Cross-Shard-Handoff + RPC-and-Protobuf: signed-snapshot optionals now written UNCONDITIONALLY — a second append-if-non-empty optional made digest(tier="",account=X) == digest(tier=X,account="") (length prefixes solve boundaries, not presence); instances rejected at every off-box ingress. - Persistence: an instance is never a durable location; the anchor + COALESCE. - Pack-Entity-Reference: zone `instanceable` flag, start_room respawn semantics. - Pack-Lua-Scripting: mud.zone(), mud.send_to_instance (self-only). - Orchestration: instances withheld from reserved schedule fan-out. - Sidebar + Engine-Developer-Reference index entries.

    @yfronto yfronto committed Jul 19, 2026
  • Round 31 milestone: handoff phase-2 auth, checkpoint tie-break, drain-reservation TTL - Cross-Shard-Handoff + RPC-and-Protobuf: document that Handoff.Commit/Abort are now Ed25519-signed under the cluster keypair with the digest bound to the destination shard id (#314) — the deterministic handoff_token over public inputs was forgeable, and destination binding stops a captured signed Abort from being replayed against a second destination in a split-brain race. - Persistence-and-Durability + Distributed-Systems-Model: the load freshness check breaks a state_version tie toward the Redis checkpoint (#322); a checkpoint is dumped at the pre-CAS version so it ties the row while carrying newer content — the strict > left the tier inert, so the tie-break is what makes the crash window the ~10s cadence and not the ~60s flush. - Cross-Shard-Handoff + Running-at-Scale + Distributed-Systems-Model: the drain-target reservation TTL now outlasts the whole drain (deadline + one presence-reflect window) so a slow-but-alive drain keeps its hold while players migrate; only a crashed drainer lingers the full TTL (#334).

    @yfronto yfronto committed Jul 13, 2026
  • Round 30 milestone: reconnect/placement/drain correctness (#321/#324/#329/#331/#336) + RID re-home (#373) Round did not self-document. Closed: - Persistence-and-Durability: a live reconnect routes by an in-memory residency index BEFORE the lagging durable zone_ref (prevents a double-owned character when a link-dead resume beats the async flush of an intra-shard walk) (#321); the placement writer gets the saver's shutdown-barrier treatment so a logout tombstone isn't lost on graceful shutdown (else the tell/mail oracle reports a player on a dead shard) (#331) - Distributed-Systems-Model: the placement tombstone now carries a per-session nonce, fencing a same-shard/same-epoch relog that shard+epoch alone couldn't distinguish (#329) - Edge-and-Protocol: a fresh login bounced by a draining shard (Unavailable) is re-resolved + re-dialed to the peer instead of dropped; token-bearing re-dials never retried (extends the zero-drop drain to ARRIVING players) (#324) - Cross-Shard-Handoff: a gate-wedged player is skipped by the drain and reclaimed (client-fault straggler), not miscounted zero-drop Redirected (#336) - Entity-Component-Model: an intra-shard transfer re-homes the entity + carried subtree to fresh per-zone RIDs (the overworld 'map on some rooms' bug) (#373)

    @yfronto yfronto committed Jul 12, 2026
  • Round 27: the AdoptZone fence cluster (#316/#288/#327) The round self-documented #315 (generation fence) on RPC + Sysadmin. Closing the rest: - Cross-Shard-Handoff: new 'Zone adoption: build, confirm, or un-adopt' — the destination requires from_shard_id to be the zone's live owner and refuses before building (#316, a correctness precondition, not a security barrier since owner+gen share one read); adoption is confirmed by observing the lease flip LAND (via ClaimZone), never by the RPC returning; an unconfirmed runtime adoption is un-adopted when its confirm deadline expires, gated on an 'adopted' flag so boot zones fence rather than self-delete (#327). - Zone-Runtime-and-Actor-Model: new 'Runtime zone teardown' — UnhostZone + the per-zone actor context; the four guards (disposable-not-unowned; quiescence is not pop==0; post selects on a dead channel or the saver drainer wedges the whole shard; publish+arm under one lock) (#288). - RPC-and-Protobuf: the from_shard_id line now notes the live-owner precondition. - Running-at-Scale: the source tears down a rebalanced-away zone (no zombie per migration).

    @yfronto yfronto committed Jul 10, 2026
  • Round 26: the placement record becomes the reconnect-routing spine Track-6 residuals. Round self-documented only #276 (catch-up metrics); the rest of the engine/ops surface was stale. - Distributed-Systems-Model: NEW 'The placement record' section — zone is the routing key (ShardForZone resolves the current owner, so an offline rebalance is transparent), epoch is the fence AND the tell/mail existence key, shard routes nothing. Every residency registers a placement (incl. the intra-shard zone walk); registerPlacement accepts an equal epoch where the handoff CAS demands a strictly greater one. Clean logout writes a fenced compare-and-delete tombstone, not a delete (#320 slice 2, #70). Also: lease renewal restarts on re-adoption, closing an A->B->A split-brain (#288). - Persistence-and-Durability: close the shard-not-zone 'Known gap'; add the saver shutdown sentinel barrier (#282). - Edge-and-Protocol: world-side Send watchdog — keepalive can't see a gate whose transport acks PINGs while its application stopped reading (#274). - Orchestration-and-Directors: a runtime-adopted zone's scope replica is seeded before the zone is exposed (a full-map replace would clobber newer state) (#280). - Cross-Shard-Handoff / Running-at-Scale / DistSys: drain-target reservations now carry per-field, server-timed expiry (#284). - Running-at-Scale: drop the self-contradicting 'deliver_lag_ms silent' gap; the gate row now describes zone-keyed reconnect routing.

    @yfronto yfronto committed Jul 10, 2026
  • Scaling docs: reflect shipped load-aware placement rebalancer Track 6 drain/scale landed: occupancy-weighted rebalancing that executes (director issues rebalance-drain directives with cooldown + locality colocation) and director-owned, reservation-based drain-target selection. Updates the now-stale 'advisory-only / naive drain-select' language on Running-at-Scale, Distributed-Systems-Model, Cross-Shard-Handoff, and Orchestration-and-Directors.

    @yfronto yfronto committed Jul 9, 2026
  • Engine Dev: directors, scoped event bus, handoff, distsys model - Orchestration-and-Directors: director-as-actor, per-scope leader election, scope hierarchy, worked boss loop; unauthored-director-script flag - Scoped-Event-Bus: two lanes, write-up/read-down, apply-once watermark, the no-cross-scope-budget nuance - Cross-Shard-Handoff: two-phase Prepare/Commit, signed snapshot + tier carry, graceful drain, gate redirect/replay - Distributed-Systems-Model: single-writer spine, leases, placement, the one-zone-one-core ceiling + honest scaling gaps

    @yfronto yfronto committed Jul 9, 2026
  • Scaffold wiki + author About and Getting Started - 48-page skeleton across all audiences (flat files + grouped _Sidebar) - Every page carries an Audience + Status banner (Ready/Draft/Placeholder) - About: Overview, History of MUDs (Ready) - Getting Started: Getting-Started, Running-Locally, First-Admin-Setup, Content-Packs-Intro (Ready) — dev bypass, local GitHub OAuth, bootstrap admin, promote/demote, content-pack model, all grounded in code - SRD5/WoW/D6 pack pages are honest placeholders (packs not yet built)

    @yfronto yfronto committed Jul 9, 2026