Skip to content

v7.200.17

Choose a tag to compare

@github-actions github-actions released this 02 Nov 02:03
bf65b6e

Shell write-gate escape classes + knowledge family migration recency (#3518, #3521)

Closes #3518 and #3521 (collated).

#3518 — five shell write-gate escape classes (security)

  • Quoted process-substitution operands no longer escape the declared
    scope.
    The parse-failure string fallback dequotes exactly-pair-quoted
    operands (matching the AST branch) and withdraws — fail-closed — every
    unresolvable shape: fragmented/unbalanced quoting, quoted destinations
    containing whitespace, backslash-escaped-space operands, multi-redirect
    stages where the lazy capture only sees the last target, and any
    procsub write on a line whose text carries a cd/pushd <target> word
    (then/do/brace-led forms included).
  • An unprovable cd is judged at BOTH landings. The tracked walk models
    the cd failing (cwd kept); the pessimistic walk models it succeeding at the
    bash-true target (Git Bash /x/... drive spellings mapped, landings
    propagated through && chains and control-flow bodies) — the gate requires
    every landing to pass. A cd whose target the same command creates
    (mkdir -p zz && cd zz && …), grouped/brace/if-body cd forms, and
    Git-Bash drive paths all deny when the success landing is outside the
    declared scope. Known edge: a failing cd before && now conservatively
    blocks the never-executed right-side write — fail-closed on an ambiguous
    shape.
  • The process-substitution fallback after a cd no longer resolves at the
    starting cwd.
    cd ../.. && true >(truncate -s 0 src/f) is denied like
    its procsub-free twin (withdrawn, fail-closed). Known edge: this
    over-blocks lines that mix a working cd with a parse-failing process
    substitution — bounded to that shape.
  • Same-literal writes separated by a cd are each scope-checked at their
    own resolved path.
    The #3347 occurrence-suffix is widened to every
    duplicate write (not just capability evals), so cd src && echo x > f; cd ../.. && echo PWN > f is denied at the second occurrence. Detector
    surfaces now show per-occurrence operators (> #2); same-cwd duplicates
    resolve identically at the gate.
  • Parenthesis-led destructive segments reach the live gate arms. The
    rd/rmdir/del (cmd.exe), rm (POSIX), and Remove-Item (PowerShell) anchors
    tolerate a leading open-paren — glued (rd x /s), spaced ( rd x /s ),
    and POSIX brace groups { rm -rf x; } — closing the reversed wedge where
    the classifier denied (rd ../../outside /s) while the live gate
    admitted it.

#3521 — knowledge family migration (data-loss class, config-ungated)

  • Same-id merges are recency-aware. mergeStoreEntries and the
    append-union merge keep the NEWER version on a same-id collision
    (updated_at with revision tiebreak on the store; updated_at on
    unactionable; quarantined_at on quarantined), destination-newer
    unchanged, no-signal collisions keep the destination and are reported
    (superseded/noSignal counts added to the migration report). Within-side
    duplicate ids select the effective (newest) row. The #2958 stale-wins
    class is closed in the knowledge engine.
  • Real producer rows are keyed and survive link/unlink.
    knowledge-events.jsonl rows key by event_id (legacy top-level id
    still keys); knowledge-application.jsonl rows use a canonical whole-row
    identity (bounded sha1, knowledgeId required) so distinct application
    outcomes for one entry survive while exact retries dedupe. A fresh link no
    longer silently drops source event rows and unlink copy-back no longer
    aborts the whole family migration at the pre-commit validator.

Known edges

The string fallback remains a heuristic tokenizer (the #3491/#3088 detector
programs track the deeper rewrite); this change closes the five frozen
escape classes plus the feedback-round escapes (mkdir-then-cd, grouped cd,
drive-phantom paths, torn procsub operands) and widens fail-closed
withdrawal to ambiguous quoting and post-cd fallback writes. Bare cd,
cd ~, and popd-adjacent directory changes remain untracked (#3491
program).


Stale store locks recover on restart (#2967, Workstream J slot J20)

A hard kill (crash, host death, kill -9) mid-write used to leave the
evidence/plan-ledger lock and the events-store lock held by a process that no
longer existed: every subsequent write stalled for the full 60 s lock budget
(EvidenceLockTimeoutError) or failed fast (CORE_EVENT_STORE_LOCKED) until
the 5-minute TTL stole the artifact — on the very stores the resumability
promise rests on.

The lock substrate now carries the owner identity INSIDE the lock file
(.swarm/locks/<sha256>.lck, wx-created with a pid/start-time/nonce payload),
and contention runs one fenced liveness break: a lock whose owner process is
provably dead (ESRCH) is broken in milliseconds with one bounded telemetry
event (store_lock_stale_broken) and a per-target cooldown; a LIVE owner is
never touched, paused or not — strictly safer than the old TTL steal, which
broke any 5-minute-old lock with no liveness check at all. Breaking is
serialized by rename-atomic markers, releases are identity-fenced (a late
release can never delete a successor's lock), and ownerless legacy locks keep
today's TTL-only behavior through the same protocol. Lane locks migrate onto
the identical substrate (one mutually exclusive protocol per target); the
events store adopts the same payload-writing two-phase acquire, dropping its
raw mtime stale-break. The PID-reuse corner (a dead holder's pid recycled by
a long-lived unrelated process) is report-only via /swarm config doctor --integrity — surfaced, never auto-removed. One disclosed upgrade window:
during a mixed-version deployment, an OLD-code writer that starts after this
version's legacy probe completes holds only the legacy proper-lockfile
artifact, so mutual exclusion between that old writer and a new-code holder
of the same target is best-effort for the duration of the upgrade.

Review feedback round (PR #3533 post-review hardening)

The independent review round hardened the substrate further:

  • Stale recovery now covers every lock target. The fenced breaker was
    wired only into the evidence/events wrappers; a dead owner's .lck on a
    plan/save target wedged that target permanently. The plain acquire path
    now probes the payload on contention and runs the same fenced, never-
    break-live-pid breaker for dead owners (cooldown-bounded).
  • Lane identity is projected again. listActiveLocks reads the .meta
    sidecar so the lean lane gates (phase-ready / task-completion) see held
    lane locks; a lane release keeps the meta when the fence refuses (no more
    orphaned live locks without bookkeeping) and releases pre-upgrade legacy
    lane locks through their .lock pairing.
  • Break markers are atomic and swept. The break right is published via
    a unique candidate file + no-replace hardlink (a mid-publish marker can
    no longer be read half-written and taken over), unparsable markers are
    never taken over, and cleanupExpiredLocks plus the doctor sweep aged
    .breaking/.take. residue; the cooldown budget is only consumed once a
    break is actually attempted, and expiry is test-pinned.
  • The doctor sees the events lock. .swarm/events.lock (and break
    markers) are classified by /swarm config doctor --integrity; the report
    strips control characters from echoed names, skips symlinks, bounds its
    reads/walks, mirrors the strict v1 payload validation, and its torn-tail
    window now covers the legal 256 KiB max line.
  • Telemetry emit path stays fast. The OTLP spool write caps its rename
    retry schedule at ~85 ms (was up to 385 ms of synchronous sleeps on the
    emitting thread), and terminal failures now assert the no-clobber
    invariant in tests driven through the real retry helper.

Crash-consistency ratchet: safer Full-Auto/OTLP writes, Check 10, doctor --integrity (#2969, Workstream J slot J21)

Two durable-state writers lost the canonical-writer contract on failure:
src/full-auto/state.ts renamed its temp with a bare renameSync (no retry
on Windows-transient EPERM/EBUSY, no temp cleanup on failure) and the OTLP
exporter's "fallback" clobbered the canonical spool/state file
non-atomically, swallowed the error, and leaked the temp. Both now ride one
shared bounded-retry helper (renameSyncWithRetry, exported from
src/utils/atomic-write.ts); the non-atomic fallback is gone; terminal
failures clean their temp and are counted once at the record-owning boundary
(write_retry_exhausted vs a separate state_write_failed counter), and the
fail-open contract is preserved end to end.

bun run check:invariants gains Check 10, the raw-rename/raw-rm ratchet
(scripts/raw-write-allowlist.txt): every remaining bare renameSync in
src/** and raw rm in the db-touching test surface is frozen behind a
per-file count quota that can only shrink (growth in either dimension
without an # APPROVED-NEW: marker fails; dead entries fail; the initial
freeze is bootstrap-gated). Burn-down is tracked by the shrinking count on
epic #2945. (The issue's "Check 9" number was already taken by #2946 — the
ratchet lands as Check 10.) /swarm config doctor --integrity now reports —
never repairs — torn ledger/events tails, temp-artifact leftovers, stale
locks (including the report-only live-pid sub-class), orphaned .bak files,
and same-id memory-family divergences, with per-category counts and a total.
docs/plan-durability.md documents the flag and a crash-consistency fsync
policy table whose flip rule requires a measured Windows cold-FS latency
first; this PR changes no fsync behavior.


fix(skills): bind the critic dispatch to the harness's pinned cross-model critic

Summary

  • Added a "Critic role binding (all profiles)" paragraph to the Phase 8 Critic Challenge of .opencode/skills/swarm-pr-review/SKILL.md — the critic dispatch must resolve to an ELIGIBLE pinned critic (not the reviewer's model family where identity is exposed); a same-family pinned critic still serves under the degraded disclosure; the generic fallback is Profiles B/C only (no pinned type on the roster, or a Hard Rule 14 failed dispatch), and a degraded gate restricts the verdict to REQUEST_CHANGES or INCOMPLETE
  • The policy binds harness adapters declaratively (mechanical enforcement is tracked in #3537): a condensed/installed skill copy whose routing note names a generic agent for this role without the pinned-first rule is defective and must be corrected, not followed

User-facing changes

Binding routing-policy change to the swarm-pr-review skill text (prose-only; no runtime code): orchestrators following installed condensed copies no longer get a generic-agent routing note to follow for the critic role; an ineligible (same-family) pinned critic now requires the degraded-independence disclosure instead of passing silently; a degraded critic gate restricts the verdict to REQUEST_CHANGES or INCOMPLETE. Installed copies change only when someone corrects them — enforcement is #3537's scope.

Migration notes

None required. Installed copies that carry a generic-agent critic routing note should be corrected to the pinned-first policy (the canonical paragraph is the reference wording).

The feedback round hardened the binding (eligible-type selection, same-family disclosure, Hard Rule 14 failure definition + BLOCKED exception, identity-not-exposed rule, Profile A carve-out, exhaustive gate slot + provenance bullet), moving the canonical's ratchet baselines pinning .opencode/skills/swarm-pr-review/SKILL.md to 1716 in tests/unit/skills/progressive-disclosure-ratchet.test.ts and tests/unit/skills/swarm-pr-review-partial-settlement-2859.test.ts; the new tests/unit/skills/critic-role-binding-pin.test.ts text pin guards the binding clauses independently of the ratchet. The paragraph is binding policy that installed copies must carry inline, so it stays in the canonical rather than moving into references/; the skill's progressive-disclosure debt grows by the same lines.