guest sdk: large-room callbacks — roster-epoch ctx mode + declared heartbeats + Rust parity

.changeset/large-room-callbacks.md

@@ -0,0 +1,30 @@
+---
+"kit": minor
+---
+
+Large-room callbacks: negotiated roster-epoch ctx encoding, game-declared
+heartbeats, and the Rust mirror of the 1024-player baseline work.
+
+- **Ctx roster-epoch mode** (`GameMeta.CtxFeatures: kit.CtxFeatRosterEpoch`):
+  the host sends the full member list only when the roster changes (with a
+  `u32` epoch) and a 6-byte unchanged marker otherwise — removing the
+  O(members) encode/decode/copy from every callback (~100KB per input at
+  1000 players, the dominant large-room input cost and the residual
+  `-gc=leaking` leak). Sentinel member-counts `0xFFFE`/`0xFFFF` (ABI.md
+  §4.1); legacy guests stay byte-identical; ABI major stays 2. Both SDKs
+  decode the sentinels and keep an epoch-aware roster cache (zero member
+  allocations on the unchanged form); the legacy byte-skim cache is retained
+  for pre-feature hosts.
+- **`GameMeta.HeartbeatMS`**: declare your wake cadence (0 = default;
+  validated 20..1000 at meta encode). Host precedence: admin
+  `host.heartbeat_ms` > declaration > 50ms default. `shellcade-kit play`
+  honors the declaration locally.
+- **Rust crate parity**: mirrors both meta fields + the sentinel decode +
+  the epoch roster cache (`Rc`-shared members), and ALSO picks up the
+  v2.5.0 Go-only baseline work it missed: `ROSTER_CAP` 16 → 1024 with
+  lazily-allocated per-slot baselines and allocated-only broadcast
+  reconcile. Golden vectors pin the Go/Rust meta encodings byte-identical.
+- New trailing meta section (`u32 ctxFeatures` + `u16 heartbeatMS`) after
+  the config-spec section, presence-guarded both directions; `GUIDE.md`
+  gains the large-room authoring section (heartbeat + feature declaration +
+  render-on-change dirty tracking).

ABI.md

@@ -119,6 +119,32 @@ u16 memberCount       on `leave`, includes the departed entry
 u8  settled           0/1
 ```
 
+**Roster-epoch member-section forms (minor addition).** For a guest whose
+meta declares `CtxFeatRosterEpoch` (§4.2), the host MAY replace the member
+section with one of two sentinel forms keyed on `memberCount` (real rosters
+are capped far below the sentinels, so the three forms are unambiguous):
+
+```
+memberCount 0x0000..0xFFFD   legacy full roster (exactly the layout above)
+memberCount 0xFFFE           full roster at an epoch:
+                               u32 rosterEpoch · u16 realCount · members as above
+memberCount 0xFFFF           roster unchanged since an epoch:
+                               u32 rosterEpoch   (no member data)
+```
+
+Lifecycle: the host holds a per-instance roster epoch, bumped on every roster
+mutation (join, leave, index shift). It sends the `0xFFFE` full form when the
+epoch differs from the last full form sent to THIS instance — which includes
+the **first callback after any instantiation or hibernation restore** (epoch
+state is ephemeral host memory, never snapshotted) — and the 6-byte `0xFFFF`
+unchanged form otherwise. The guest treats a full form as authoritative
+whenever received (re-cache, adopt the epoch); an unchanged form whose epoch
+differs from the guest's cached epoch is a host fault — the guest logs once,
+keeps its cached roster, and degrades (it must never trap on it). Guests that
+do not declare the feature receive ONLY the legacy form, byte-identical to
+prior revisions. The roster epoch and the frame-delta epoch (§4.6) are
+independent counters.
+
 ### 4.2 Meta
 
 ```
@@ -133,6 +159,8 @@ u16 configSpecCount                                              (trailing; see
   per spec: str key · str title · str description
             · u8 type (0 text · 1 number · 2 bool · 3 json)
             · str default ("" = not declared) · str schema ("" = none; json only)
+u32 ctxFeatures       trailing large-room section (see below); bit 0 = CtxFeatRosterEpoch
+u16 heartbeatMS       0 = no declaration
 ```
 
 `slug` must be non-empty; the host refuses artifacts whose slug or version it
@@ -155,6 +183,19 @@ intended to be a JSON Schema document — compilation and enforcement are a host
 concern). The Go SDK enforces these rules at `meta()` encode time, and
 `wire.ValidateConfigSpecs` is the shared rule set for decoders.
 
+**Large-room section (minor addition).** A second trailing section after the
+config-spec section: `u32 ctxFeatures` (a bitset of negotiated callback
+encodings; bit 0 = `CtxFeatRosterEpoch`, see §4.1) then `u16 heartbeatMS`
+(the game's preferred wake cadence; 0 = no declaration). Presence-guarded
+exactly like the config-spec section: a payload ending after the config-spec
+section is a valid older meta with zero values; older hosts ignore the
+trailing bytes; hosts ignore feature bits they do not implement. Encoders
+that know the section always write it. SDKs reject undefined feature bits
+and a `heartbeatMS` outside 0 ∪ [20, 1000] at `meta()` encode time. The host
+resolves the wake heartbeat at room creation as: admin `host.heartbeat_ms`
+config > declared `heartbeatMS` > platform default (50ms), clamped to
+[20ms, 1000ms] — a declaration is authoring intent, never authority.
+
 ### 4.3 Frame (the delta container and its cell)
 
 A frame is delivered as a **frame-delta container** (§4.5), a variable-length

GUIDE.md

@@ -383,6 +383,55 @@ The native runner also rides terminal resizes: shrink the window below 80×24
 and it shows a "terminal too small" notice, then repaints your game the moment
 you grow it back — the same letterboxing the arcade does over SSH.
 
+## Large rooms: 100+ players in one room
+
+The SDK supports rooms of up to 1024 players, but a large room only stays
+inside the wake budget if the game follows three disciplines:
+
+**Declare your heartbeat.** A roguelike or board game does not need the 50ms
+default. Declare your real cadence and the platform honors it (an admin
+override always wins):
+
+```go
+func (Game) Meta() kit.GameMeta {
+    return kit.GameMeta{
+        // ...
+        HeartbeatMS: 100, // gentle tick: 10 wakes/sec
+    }
+}
+```
+
+**Declare the roster-epoch feature.** By default every callback payload
+carries the full member list. In a 1000-player room that is ~100KB per
+input; with the feature, an unchanged roster costs 6 bytes:
+
+```go
+CtxFeatures: kit.CtxFeatRosterEpoch,
+```
+
+The SDK handles the rest. One contract to know: the slice from
+`r.Members()` is valid for the duration of the callback — copy any `Player`
+you keep, and key long-lived state by `AccountID` (you already should).
+
+**Render on change, not on wake.** Composing and sending every player's
+frame on every wake is the single largest cost in a big room — and most
+frames are identical to the last. Track per-player dirtiness and skip clean
+viewports:
+
+- re-compose a player's view only when THEY moved, something moved within
+  their visible window, or their HUD line changed;
+- throttle ambient HUD clocks to ~1Hz — a per-wake counter in the HUD forces
+  a nonzero delta for every player on every wake, defeating the delta
+  encoder;
+- at typical input rates expect only ~10–30% of viewports dirty per tick —
+  a 3–10× cut in wake cost.
+
+The frame-delta layer already makes CLEAN-but-resent frames cheap on the
+wire; dirty tracking makes them free in CPU too. Allocation discipline
+matters as much as ever under `-gc=leaking`: compose into a reused
+`kit.Frame`, write cells directly, avoid per-player-per-wake string
+building.
+
 ## Smoke scripts: scripted screens
 
 Every catalog game ships a `smoke.yaml` next to its source: a small

internal/game/codec.go

@@ -40,6 +40,15 @@ type callContext struct {
 var (
 	rosterCache      []Player
 	rosterCacheBytes []byte
+
+	// Roster-epoch mode state (games declaring CtxFeatRosterEpoch): the epoch
+	// the cached roster was decoded at. epochMismatch flags a host-fault
+	// unchanged-form whose epoch didn't match the cache (decodeCall logs it);
+	// epochMismatchLogged keeps that warning to one line per instance.
+	rosterCacheEpoch    uint32
+	rosterCacheEpochSet bool
+	epochMismatch       bool
+	epochMismatchLogged bool
 )
 
 // decodeCtx decodes a CallContext and returns the reader positioned at the
@@ -54,39 +63,52 @@ func decodeCtx(b []byte) (callContext, *wire.Rd, bool) {
 	c.cfg.Capacity = int(r.U16())
 	c.cfg.MinPlayers = int(r.U16())
 
-	// Skim the member section (count + per-member strings) without decoding,
-	// to find its extent for the cache compare.
-	start := r.Off
-	n := int(r.U16())
-	for i := 0; i < n && !r.Bad; i++ {
-		r.SkipStr() // handle
-		r.SkipStr() // account id
-		r.SkipStr() // conn
-		r.U8()      // kind
-	}
-	region := r.B[start:r.Off]
-
-	changed := rosterCacheBytes == nil || !bytes.Equal(region, rosterCacheBytes)
-	if changed {
-		rr := &wire.Rd{B: region}
-		cnt := int(rr.U16())
-		rosterCache = rosterCache[:0]
-		for i := 0; i < cnt && !rr.Bad; i++ {
-			var p Player
-			p.Handle = rr.Str()
-			p.AccountID = rr.Str()
-			p.Conn = rr.Str()
-			p.Kind = Kind(rr.U8())
-			rosterCache = append(rosterCache, p)
+	count := r.U16()
+	var changed bool
+	switch count {
+	case wire.CtxRosterUnchanged:
+		// Roster-epoch sentinel: epoch only, no member data. Reuse the cache;
+		// an epoch mismatch is a host fault — degrade (keep the cache), the
+		// single warning is emitted by decodeCall, which has a Room to log on.
+		epoch := r.U32()
+		if !rosterCacheEpochSet || epoch != rosterCacheEpoch {
+			epochMismatch = true
+			changed = true // be conservative: invalidate baselines
 		}
-		if r.Bad {
-			// Malformed member section: don't prime the cache — every
-			// malformed callback stays "changed" and decodes what it can
-			// (the pre-cache behavior). A well-formed region is ≥2 bytes
-			// (the count), so a primed cache is never nil.
-			rosterCacheBytes = nil
-		} else {
-			rosterCacheBytes = append(rosterCacheBytes[:0], region...)
+	case wire.CtxRosterFull:
+		// Roster-epoch sentinel: full roster at an epoch. Authoritative.
+		epoch := r.U32()
+		decodeMembersInto(r, int(r.U16()))
+		rosterCacheEpoch = epoch
+		rosterCacheEpochSet = true
+		rosterCacheBytes = nil // bytes cache is legacy-mode state
+		changed = true
+	default:
+		// Legacy full roster (pre-feature hosts): skim the member section's
+		// extent without decoding, memcmp against the previous callback's
+		// bytes, and re-decode only on a real change.
+		start := r.Off - 2 // include the count in the compared region
+		for i := 0; i < int(count) && !r.Bad; i++ {
+			r.SkipStr() // handle
+			r.SkipStr() // account id
+			r.SkipStr() // conn
+			r.U8()      // kind
+		}
+		region := r.B[start:r.Off]
+		changed = rosterCacheBytes == nil || !bytes.Equal(region, rosterCacheBytes)
+		if changed {
+			rr := &wire.Rd{B: region}
+			decodeMembersInto(rr, int(rr.U16()))
+			if r.Bad {
+				// Malformed member section: don't prime the cache — every
+				// malformed callback stays "changed" and decodes what it can
+				// (the pre-cache behavior). A well-formed region is ≥2 bytes
+				// (the count), so a primed cache is never nil.
+				rosterCacheBytes = nil
+			} else {
+				rosterCacheBytes = append(rosterCacheBytes[:0], region...)
+			}
+			rosterCacheEpochSet = false // epoch state is sentinel-mode only
 		}
 	}
 	c.members = rosterCache
@@ -95,6 +117,20 @@ func decodeCtx(b []byte) (callContext, *wire.Rd, bool) {
 	return c, r, changed
 }
 
+// decodeMembersInto re-decodes the member list into the shared rosterCache
+// backing array (the only place member strings are allocated).
+func decodeMembersInto(r *wire.Rd, n int) {
+	rosterCache = rosterCache[:0]
+	for i := 0; i < n && !r.Bad; i++ {
+		var p Player
+		p.Handle = r.Str()
+		p.AccountID = r.Str()
+		p.Conn = r.Str()
+		p.Kind = Kind(r.U8())
+		rosterCache = append(rosterCache, p)
+	}
+}
+
 // encodeMeta packs GameMeta for the meta export.
 func encodeMeta(m GameMeta) []byte {
 	wm := wire.Meta{
@@ -132,6 +168,16 @@ func encodeMeta(m GameMeta) []byte {
 	if err := wire.ValidateConfigSpecs(wm.ConfigSpecs); err != nil {
 		panic("kit: invalid GameMeta.Config: " + err.Error())
 	}
+	// Large-room trailer: ctx-feature bits + declared heartbeat, validated
+	// under the same fail-fast posture.
+	if m.HeartbeatMS < 0 || m.HeartbeatMS > 0xFFFF {
+		panic("kit: invalid GameMeta.HeartbeatMS: out of range")
+	}
+	wm.CtxFeatures = m.CtxFeatures
+	wm.HeartbeatMS = uint16(m.HeartbeatMS)
+	if err := wire.ValidateMetaTrailer(wm.CtxFeatures, wm.HeartbeatMS); err != nil {
+		panic("kit: invalid GameMeta: " + err.Error())
+	}
 	return wire.EncodeMeta(wm)
 }
 

internal/game/devrun.go

@@ -34,6 +34,17 @@ func Main(g Game) {
 	flag.Var(cfgFlag(cfgVals), "config", "KEY=VALUE per-game config (repeatable; value may be @file)")
 	flag.Parse()
 
+	// Honor a meta-declared heartbeat unless -heartbeat was given explicitly,
+	// so authors experience their declared cadence locally (the host applies
+	// the same declaration with admin-config precedence in production).
+	if hb := g.Meta().HeartbeatMS; hb > 0 {
+		explicit := false
+		flag.Visit(func(f *flag.Flag) { explicit = explicit || f.Name == "heartbeat" })
+		if !explicit {
+			*heartbeat = time.Duration(hb) * time.Millisecond
+		}
+	}
+
 	// -seed makes the whole run reproducible: a fixed RNG seed AND a virtual
 	// clock (see below). Distinguish "flag given" from "left at default 0" so a
 	// deliberate -seed 0 still goes deterministic.