fix(mol-witness-patrol): liveness lookup is exact-match on assignee, so rig-qualified bead-host owners resolve 'absent' and are false-orphaned (tk-opfsi)#210
Merged
Conversation
… rig-qualified assignees stop false-orphaning live sessions (tk-opfsi) recover-orphaned-beads resolved a bead's assignee against the session liveness map with a single exact key lookup, `.[$a] // "absent"`. The map is keyed on session-registered identities, and those routinely differ from the bead's assignee in nothing but the `<rig>/` prefix, so the lookup fell through to `absent` — which step 3 classifies as orphaned, "the owning session is gone and will never come back", for a session that is active and running. Confirmed firing on gc-z0vi2.1 (status=open, owner active); containment held only because the witness agent noticed by judgment. Neither existing guard catches it: the fail-safe trips only on a totally empty map (ours is fully populated, just mis-keyed) and the infrastructure skip does not cover pool/ephemeral bead-hosts. Unlike the sibling orphan-sweep.sh path (tk-2l13a), this step enumerates `open` beads too, so the exposure was live rather than latent — and tk-2l13a's `gc.session_id` stamp cannot reach this path, which never reads bead metadata. Takes the bead's option (b) — retry on the last path segment — over (a) rebuilding the map with both key shapes, because it is the smaller change and only ever turns `absent` into a live state. Two things the report did not cover, both from live `gc session list` output: the mismatch runs in BOTH directions in the same city (wisp bead-hosts register a bare alias while their beads are rig-qualified; pool polecats register a rig-qualified alias and may be addressed bare), so the retry normalizes both sides rather than only stripping the assignee's prefix; and stripping the prefix can collide two cities' sessions on one bare identity (gc-toolkit/gc-toolkit.furiosa and gascity/gc-toolkit.furiosa), where picking an arbitrary match could report a dead state for a live agent and re-open the bug through the fix. The retry therefore resolves a collision toward life: any matching session that is not closed/archived wins, and only when EVERY candidate owner is dead does it report a dead state. The exact lookup stays the authoritative first pass, so a genuinely dead session that hits a literal map key is still recovered. Validation: assets/scripts/liveness-lookup.test.sh (new, 30 assertions) executes the guard extracted verbatim from the formula between `liveness-lookup` markers — the same drift-proof pattern as husk-guard.test.sh — covering the confirmed-firing shape, the reverse shape, exact-match authority over a collision, unknown identities still resolving absent, order-independent collision preference across every live state in the classification list, all-dead candidates staying dead, whole-segment (not suffix) matching, and degenerate inputs. Static checks assert the formula keeps exactly one exact-match lookup, that the retry is gated on the exact pass missing, and that the classification prose no longer authorizes orphaning on the exact lookup alone. Also replayed against the live 69-key liveness map: gascity/gc-toolkit.gc-z0vi2 goes absent -> active, gc-toolkit.furiosa goes absent -> active, exact hits and genuinely unknown identities are unchanged. husk-guard.test.sh still passes (16/16); shellcheck is clean apart from the accepted info-level SC2015/SC2016 idioms. Scope: gc-toolkit's own copy of the formula (not symlinked to other rigs). The durable core fix — normalizing `live_session_match()` in gascity's orphan-sweep.sh, plus the complementary false-negative where that sweep skips unassigned beads — stays deferred to gascity bead gc-uuz79w until the fork shed settles.
Contributor
Author
|
Codex signoff (pre-open, comment-only — not an approval): COMMENT signoff for pre-open branch review. Reviewed branch: polecat/tk-opfsi Scope reviewed:
Validation:
Review artifact: /tmp/compound-engineering/ce-code-review/20260722-150015-b90f0568/report.md |
johnzook
approved these changes
Jul 22, 2026
zook-bot
added a commit
that referenced
this pull request
Jul 22, 2026
…live teardown (tk-z130v.3)
Ground gc-toolkit bead-hosts so an involuntary config-drift drain no longer
kills them. gc-bead-host.sh 'link' sets the work bead's assignee to the host's
session NAME — the assigned-work wake reason the reconciler honors across drains
(compute_awake_set has no Drained gate) — and 'unlink' / gc-helm 'takeaway
--release' clear it so a host can still be stopped. New 'backfill' grounds hosts
linked before this shipped; new/resumed hosts ground automatically via 'up'.
- tools/gc-bead-host.sh: link grounds, unlink ungrounds, + backfill migration verb
- agents/bead-host/{prompt.template.md,agent.toml}: warm-while-live model — a
grounded host stays warm while its bead has awake-demand; teardown clears the
wake reason (close/park/release) FIRST, then drains
- assets/scripts/gc-helm.sh: note that --release's assignee-clear also ungrounds
- formulas/mol-witness-patrol.toml: recover-orphaned-beads skips grounded host
beads (assignee==host_session_name) — safe-by-code, not judgment (cf #206/#210)
- tests: bead-host-binding-fixture.sh grounding assertions; host-bead-skip.test.sh
Guardrails verified (scale-check/keeper/pool/refinery/reconcile key on
gc.routed_to/branch/merge_result/own-identity, never a session-name assignee).
zook-bot
added a commit
that referenced
this pull request
Jul 23, 2026
…ke-reason-aware teardown (tk-z130v.3) (#213) ## Summary # Ground gc-toolkit bead-hosts in core (assigned-work wake reason) + wake-reason-aware teardown Parent decision: tk-z130v (read its notes — the A/B evidence and the accepted trade are there). Repo: `zookanalytics/gc-toolkit` (INTERNAL fork; standard bead→PR→Codex→ refinery flow — open a PR, do NOT self-merge). This IS an implementation bead. ## Goal (one sentence) A gc-toolkit bead-host must survive an INVOLUNTARY config-drift drain by being brought back automatically by the reconciler with its conversation intact — achieved by making the host the **assignee of its hosted bead** (the assigned-work wake reason survives the `drained` state), NOT by any fork exemption. ## Proven facts you are building on (do not re-derive; verify only where noted) - gascity core revives a stopped session that is the assignee of a bead with awake-demand. A/B proven on this city: assignee set ⇒ revived ~20s, no operator action; assignee cleared ⇒ stayed `drained`. (Mechanism: `compute_awake_set.go` assigned-work loop has no `Drained` gate; `workBeadHasAwakeDemand` = in_progress, or open+Ready.) - Assignment MUST use the session **name** (`s-<id>`), not the session id — passing a session id to `gc bd update <tk-bead> --assignee` flips the resolver to the HQ store and fails "no issue found". `s-<id>` is also what the awake-set matches on. - A grounded host does NOT idle-sleep and cannot be stopped by plain `drain-ack` while the assignment + awake-demand hold (revives ~20s). Teardown must clear the wake reason FIRST. ## The change (core of it) The dual-link already written on host bring-up is the natural home. In `tools/gc-bead-host.sh`: 1. **`cmd_link` (bring-up)** — in addition to the existing reverse/forward/lineage writes, set the **work bead's `assignee` to the session name** (`$name`, already resolved in that function). This is the grounding. Keep it idempotent (re-linking the same session must not thrash). Note the existing convention (see `gc-helm.sh` header comment: "a child bead's `assignee` is its OWNING session — the session_name"), so assignee=session_name is already an established meaning here — confirm the host bead is consistent with it. 2. **`cmd_unlink` (teardown / re-bind)** — clear the work bead's `assignee` (alongside the existing metadata clears) so the host can actually be stopped. Without this, a re-bind or cleanup leaves the old session grounded and un-drainable. 3. **`gc-helm.sh takeaway --release`** — ALREADY clears assignee (`--assignee=`) as its closing step (verified). Confirm it still does after your change and that release = ungrounded-then-stoppable. If any other operator-facing "done with this bead" path exists, it must also clear the wake reason before drain. ## The prompt/model change (do not skip — the old guidance is now wrong) `agents/bead-host/prompt.template.md` currently tells hosts to `gc runtime drain-ack` "between visits" for token economy. For a GROUNDED host that is a no-op-with-restart (revives ~20s). Update the model to ONE coherent story and make the prompt match it: - **DEFAULT (recommend): warm-while-live.** A grounded host stays warm as long as its bead has awake-demand; it is put down by ENDING the work — close the bead, or park it (defer/block so it is no longer `ready`), or `takeaway --release`. Replace the "drain-ack between visits" instruction accordingly: intentional teardown clears the wake reason (release / close / park) FIRST, then drains. - **ALTERNATIVE (only if you verify it): hold-to-sleep.** Keep the assignment but use a bounded `gc session suspend`/`held_until` to sleep between visits — the hold beats every wake reason (spike claim, `compute_awake_set.go` "Hold suppression — overrides everything"). BEFORE proposing this, VERIFY EMPIRICALLY that a config-drift occurring DURING the hold still results in eventual revival (not permanent death). This interaction is UNVERIFIED. If you cannot verify it cleanly, do NOT adopt it — ship the warm-while-live model. ## Guardrails to verify (must not regress) - **Assignee=session_name must not make the host bead look like pool/handback work.** Confirm scale_check, keeper prime-sweep (assignee-based), witness reconcile, and any pool router do NOT pick up a host bead just because it now has an assignee. (The child-bead convention suggests safe, but a HOST bead is often a decision/open bead — check it isn't treated as an unassigned handback or a routable root.) - **Migration / live hosts.** After deploy, existing live bead-hosts are still ungrounded. Provide a one-time backfill (set assignee=session_name on every live host bead from its hosts_bead link) OR document that they ground on next `open`. This matters: the exemption drop (tk-z130v.4) is gated on all live hosts being grounded, and THIS conversation's host (tk-z130v) is one of them. - **Idempotency / cross-rig.** `up` runs cross-rig with `BEADS_DIR` pinned; the assignee write must land in the bead's own rig ledger (same pinning as the existing link writes). ## Tests / verification (TDD where practical) - Offline-testable: `link` sets assignee=session_name on the work bead; `unlink` clears it; both idempotent. (gc-bead-host.sh already has offline-testable link/unlink — extend those.) - Live smoke on a THROWAWAY host (mirror tk-z130v's A/B; do not touch real hosts): open → assignee set → drain/idle → auto-revive; release/unlink → assignee cleared → drain sticks. Tear the throwaway down cleanly. - Do NOT drift real city config to test the involuntary path; the assigned-work grounding is the tested invariant and the idle-revive A/B already stands in for it. ## Deliverable A PR on `zookanalytics/gc-toolkit` implementing the above, with the prompt/model updated to the shipped sleep story, guardrails verified, and a backfill (or a documented ground-on-next-open) for live hosts. Report the PR number back on this bead. Do NOT open any upstream (gastownhall) PR. Do NOT touch the gascity fork or gc-8yr6px — that is tk-z130v.4, deliberately separate and blocked on this. ## Implementation notes Implemented (branch polecat/tk-z130v.3 @ ca40c4d, target main). Grounding + warm-while-live teardown. CORE (tools/gc-bead-host.sh): `link` sets work-bead assignee=session_name — the assigned-work wake reason compute_awake_set revives across an INVOLUNTARY drain (no Drained gate); `unlink` clears it (ungrounds → stoppable); new `backfill` grounds already-linked hosts city-wide, rig-pinned per bead. New/resumed hosts ground automatically via up→link. gc-helm `takeaway --release` already clears assignee — verified + noted it ungrounds. MODEL (agents/bead-host/{prompt.template.md,agent.toml}): warm-while-live — a grounded host stays warm while its bead has awake-demand; teardown clears the wake reason (close/park/release) FIRST, then drains. hold-to-sleep NOT adopted (unverified; a polecat must not drift live config). GUARDRAILS (verified, must-not-regress): scale-check/keeper/pool-router/refinery/reconcile never act on a session-name assignee (they key on gc.routed_to/branch/merge_result/own-identity; grounding also fails --unassigned). The witness recover-orphaned-beads scan newly sees grounded host beads, but drained/asleep already classify not-orphaned (revival scenario safe); closed the dead-host edge safe-by-code (cf #206/#210): recover-orphaned-beads drops beads grounded to their own host (assignee==host_session_name). TESTS: bead-host-binding-fixture.sh grounding/unground/idempotent/backfill (31/31 live); host-bead-skip.test.sh (5/5, extracts the witness filter verbatim). bash -n + shellcheck clean. Rebased onto origin/main incl. #205 — quiesce is orthogonal (keys gc.step_ref=mol-polecat-work.*, never host beads). DEFERRED (operator; needs a live LLM session — a polecat must not spawn/reset live sessions): live smoke on a throwaway host (open→assignee set→drain→auto-revive; release/unlink→cleared→drain sticks). Parent tk-z130v already A/B-proved revival; this ships the metadata contract that triggers it. Scope: gc-toolkit only (no upstream/gascity; tk-z130v.4 separate). PR opens via the refinery pre-open codex gate — PR number to be recorded then. Rework tk-5bygy landed on polecat/tk-z130v.3 at 358e0b7; re-review tk-knb1t dispatched (one-anchor-per-PR, tk-ynz4b). Rework tk-v369i landed on polecat/tk-z130v.3 at 9fd8e47; re-review tk-wc3gf dispatched (one-anchor-per-PR, tk-ynz4b). Rework tk-0gnq9 landed on polecat/tk-z130v.3 at cc62091 (rebased onto origin/main 4b09bd5 — resolved the assets/scripts/gc-helm.sh cmd_clear conflict with landed tk-xypcy #211, preserving the codex signoff fixes); re-review tk-q6zwf dispatched (one-anchor-per-PR, tk-ynz4b). Gates at cc62091: bash -n clean; shellcheck clean at warning+ (info-only SC2015 x41 on the A && ok || bad assertion idiom, SC2016 x4). ## Refinery handoff - Issue: `tk-z130v.3` (task, P1) - Source branch: `polecat/tk-z130v.3` - Target: `main` - Codex signed off pre-open at `cc620913`; PR opened codex-green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sibling to tk-2l13a — same root cause, DIFFERENT code path, and this one is NOT latent.
tk-2l13a fixes the
orphan-sweep.shpath by stampinggc.session_idon ownedbeads (that script's
session_bead_candidates()reads bead metadata). Thatstamp cannot reach this path, so tk-2l13a alone leaves live bead-hosts exposed.
The defect (verified at file:line, gascity witness + mayor independently)
rigs/gc-toolkit/formulas/mol-witness-patrol.toml, recover-orphaned-beads step:That is an exact key lookup of the raw assignee string against a
session-derived map. It never reads bead metadata. Bead-host sessions
register a bare alias while their beads carry a rig-qualified assignee,
so the lookup always falls through to
// "absent":absentis then classified orphaned — "the owning session is gone and willnever come back" — for a session that is active and running.
Why this is worse than tk-2l13a's exposure
orphan-sweep.sh) collects onlyin_progressbeads, so itsexposure is latent until a bead-host takes work in-progress.
mol-witness-patrol.toml:135-136):gc bd list --status=in_progress --json --limit=0
gc bd list --status=open --json --limit=0
So
openbeads are in scope and the exposure is live right now.absentthis cycle. The gascity witness recognised it and did NOT act —containment held by judgment, not by code.
Why the existing guards do NOT catch it
("empty liveness map with N live sessions — skipping orphan recovery").
Ours is fully populated, just mis-keyed, so it never fires.
Bead-hosts are pool/ephemeral work identities, so they are not skipped.
Fix
Normalise identity in the lookup instead of requiring an exact match. Either:
(a) build
LIVENESS_MAPwith BOTH the bare alias and the rig-qualified form askeys, or
(b) on an
absentresult, retry the lookup on the assignee's last path segment(
gascity/gc-toolkit.gc-z0vi2->gc-toolkit.gc-z0vi2) before classifying.(b) is the smaller change and is fail-safe in the right direction — it can only
turn
absentinto a live state, never the reverse.Acceptance
With a bead-host live and a bead assigned to it (status
openAND again within_progress), a recover-orphaned-beads pass must classify it live, notorphaned. Regression-guard the rig-qualified-vs-bare case explicitly.
Scope note
gc-toolkit-owned formula, so this can land while the gascity fork shed is
running. The durable core fix (normalising
live_session_match()in gascity'sorphan-sweep.sh) is deliberately deferred until the shed settles — seegascity bead gc-uuz79w, which carries the full evidence plus the complementary
FALSE-NEGATIVE in the same sweep (it skips unassigned beads entirely).
Reported by gascity/gc-toolkit.witness on tk-2l13a; re-verified at file:line by mayor.
Implementation notes
Implemented: liveness lookup in mol-witness-patrol recover-orphaned-beads now normalizes identity — exact lookup stays the authoritative first pass, then one retry on the last '/'-separated segment, resolving bare-form collisions toward life. Live replay against the real 69-key map: gascity/gc-toolkit.gc-z0vi2 absent->active, gc-toolkit.furiosa absent->active; exact hits and unknown identities unchanged. New assets/scripts/liveness-lookup.test.sh (30 assertions, extracted verbatim from the formula markers) passes; husk-guard.test.sh still 16/16. Branch polecat/tk-opfsi @ e9731cb.
Re-offer verified (gc-toolkit.furiosa, new session after lx-fjnq1 died): work already submitted — origin/polecat/tk-opfsi @ e9731cb matches local handoff, diff = formulas/mol-witness-patrol.toml (+49/-8) + assets/scripts/liveness-lookup.test.sh (new, regression-guards the rig-qualified-vs-bare case per acceptance). Bead correctly parked with refinery (target=main, gc.routed_to cleared). No re-handoff, no re-implement, no close. Cause of re-offer: graph.v2 husk tk-z57tr's load-context step tk-1ta4k never closed (inline execution), so the pool re-offers it while this bead stays open.
Re-offer #2 verified (gc-toolkit.furiosa): origin/polecat/tk-opfsi @ e9731cb, ahead 1 / behind 0 vs main, files = formulas/mol-witness-patrol.toml (+41/-8) + assets/scripts/liveness-lookup.test.sh (+188). Bead correctly parked with refinery at merge_result=pre_open_gate (no PR yet is CORRECT for pre-open). No re-handoff, no re-implement, no close. ESCALATED ONCE to gc-toolkit.witness (mail lx-wisp-9jsfk) to retire husk root tk-z57tr and de-route never-closed step tk-1ta4k. A future re-offer should verify silently and drain — do NOT re-escalate, do NOT re-implement.
CORRECTION (same session, gc-toolkit.furiosa): the preceding note's claim 'parked with refinery (assignee=refinery)' was a stale snapshot. Dolt history (tk.dolt_history_issues) shows the refinery parked this anchor at 14:55:22 (commit a9gmlp76): assignee CLEARED deliberately, metadata gained check_set=codex + merge_result=pre_open_gate + merged_target=main, after dispatching pre-open codex review bead tk-iqlir (blocks tk-opfsi, claimed 14:55:33 by gc-toolkit__polecat-codex-lx-kz313). Empty assignee + empty gc.routed_to is the CORRECT pre-open park shape here — the anchor is held by the check-set marker and the live review bead's blocks edge, not by an assignee, and it closes on merge via refinery reconcile. I briefly re-assigned it to the refinery at 14:57:31 (misreading the empty assignee as a strand) and have reverted that. Do NOT re-assign this anchor while tk-iqlir is open.
Refinery handoff
tk-opfsi(bug, P1)polecat/tk-opfsimaine9731cb0; PR opened codex-green.