You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Add a maintainer skill-behavior instrument: 12 fixtures in six families (evidence before
completion, sensor adequacy, resume freshness, authority, unknown effects, independent
verification), each with a development and a reserved case grounded in existing corpus IDs and
system invariants. Oracles read final disk state, sensor stamps, Git-ignored sentinels, and a
hidden truth table; a test proves every oracle accepts compliant behavior and flags the scripted
violation as critical.
Rebuild npm run saf:sandbox-campaign on those oracles: an exit code is an observation, never a
verdict; a forbidden effect fails even after a timeout; timeouts, budget stops, and missing
capabilities never count as PASS. Cases come from the manifest, the report version follows package.json, a Claude Code adapter joins Codex and Cursor with isolated profiles, and an
offline stub host exercises the pipeline without model calls.
Add npm run saf:skill-behavior:compare, an offline baseline/candidate comparison that refuses
unpaired, duplicated, or mixed data and never aggregates across hosts or models.
Document the protocol in docs/skill-behavior-evaluation.md, correct docs/inspirations.md (SAF
contract fields live in saf-contract.yml), update the ROADMAP direction, and clarify that
rewind and rollback are host capabilities.
Fix doctor ignoring its homeDir option when resolving user skill directories, installation
intent, contracts, and autonomy checks.
Run the first real skill-behavior campaign (12 cases × Claude Code 2.1.287 and codex-cli 0.162.1
× 1 trial on the 8.3.0 bundle): 24/24 PASS. Pilots first exposed and fixed instrument defects:
a Codex --sandbox/--approve-for-me conflict, closeout status parsing, Claude Code bundle
placement (.claude/skills), and headless approval denials, now recorded as a missing
capability. Claude Code runs Bash inside its own sandbox; positive claims on a real host need a
sensor command in the harness-captured transcript, so a forged sensor stamp fails.
Mark skill loading as manually verified for those host versions. One trial and no no-SAF control
measure neither variance nor SAF's effect, and no skill instruction changed, so no behavior
improvement or cost claim is made. No new command, flag, runtime dependency, telemetry, or
public schema.