Skip to content

Releases: wandercom/signet-eval

v3.13.0 — Claude hook recovery and guard repairs

Choose a tag to compare

@jmc-wander jmc-wander released this 24 Sep 17:33
53aa4ac

Repairs Claude function-hook recovery and error reporting, and releases the optional Claude/Kindex integration.

  • Hooks recheck current owner state instead of retaining startup conflicts. Confirmed disablement is neutral; repaired owners recover without a new registration. Active tool evaluation failures still fail closed.
  • Recovery prompts remain available, with an explicit warning when sanitation is unavailable.
  • Sanitized policy refusals and command errors reach the agent instead of being replaced by tool_result_unavailable; failed commands are not retried.
  • Binary self-protection checks executable references rather than any occurrence of the repository name. Follow-up review corrections protect ordinary wrapper invocations and the running owner executable at custom or renamed install paths.
  • Includes rmcp 2 compatibility and PR #15’s exact Claude 2.1.274/2.1.280 qualification and independent installer tests.
  • Aligns Cargo, Claude plugin, Kindex, and MCP metadata at 3.13.0. The separate policy change in PR #16 is excluded.

Validation: 259 Rust tests in each feature configuration, 56 recovery cases, 8 host-error cases, package/publish dry run, plugin validation, and isolated real-host probes on both accepted Claude builds with a synthetic provider. These probes do not attest existing sessions or a production provider.

Upgrade the binary and reinstall its embedded optional adapter together; pairing revision is claude-functions-2.1.280-v1. Installing a package does not activate the optional adapter.

See the incident report in docs/incidents/2026-09-24-claude-adapter-lockout.md for the diagnosis and evidence boundaries.

Released from 53aa4acbe3cdbae965aab32bfdd04bfa703bf99b after Linux and macOS CI passed on that exact final-main commit: https://github.com/wandercom/signet-eval/actions/runs/36034011996

Merged PRs: #15, #19, and #20.

v3.12.2

Choose a tag to compare

@jmcentire jmcentire released this 28 Aug 03:17

Antigravity permissions-adapter repair.

  • Consumes Antigravity's native nested toolCall payload and emits native top-level allow/deny decisions.
  • Normalizes command, file, multi-edit, and MCP calls into Signet's existing canonical policy model.
  • Makes native MCP wrapper identity authoritative, rejects spoofed identity fields, and fails closed on malformed wrappers.
  • Ships an Antigravity hook configuration plus README and project-site setup documentation.
  • Aligns Cargo, Kindex, MCP Registry, and Claude plugin metadata on v3.12.2.
  • Passes 231 tests, formatting, lint, crate packaging, and crates.io publish dry-run verification.

v3.12.1

Choose a tag to compare

@jmcentire jmcentire released this 01 Aug 19:14

Durable safety-default release.

  • Ships the GitHub remote-owner identity guard and embeds its check for atomic first-use installation.
  • Restores hard denial of ephemeral Task* state in favor of Kindex tasks.
  • Requires same-session Kindex engagement before direct edits; missing distinct host session identity fails closed.
  • Reconciles current compiled built-ins over stale machine policy snapshots while preserving distinct host rules and user overrides.
  • Passes 218 tests, shellcheck, release build, and crate package verification.

v3.12.0

Choose a tag to compare

@jmcentire jmcentire released this 13 Jun 06:08

Add Antigravity and OpenCode hook adapters, and change prefer_persistent_task_store default policy to Ask

signet-eval v3.11.1

Choose a tag to compare

@jmcentire jmcentire released this 29 May 01:14

Fixed

  • Scoped preflight state to real client chat/session identifiers before falling back to SIGNET_SESSION.
  • Hook evaluation now extracts session identifiers from client payloads for preflight, pause, and disable checks.
  • Session-scoped preflights no longer deactivate global preflights or preflights from other sessions.

Verification

  • cargo test: 156 unit, 9 CLI integration, 38 hook integration passed.
  • cargo publish --dry-run passed.
  • Published signet-eval 3.11.1 to crates.io.

signet-eval v3.11.0

Choose a tag to compare

@jmcentire jmcentire released this 16 May 00:04

signet-eval v3.11.0

Adds advisory INJECT rules for probabilistic behavior nudges near tool-call time, without changing deterministic authorization semantics.

Added

  • INJECT decision variant evaluated in a separate post-auth pass.
  • Recency-weighted trigger modes: constant, step, linear, and exponential.
  • Payload sources: inline text, text_file under ~/.signet/injections/, and HMAC-signed allowlisted from_command output.
  • Template substitutions: {tool_name}, {cwd}, {date}, {matched_param.X}.
  • signet-eval injections and signet-eval inject-test <rule>.
  • examples/inject_examples.yaml.

Fixed

  • block_rm no longer false-positives on words/paths containing rm as a substring, such as drone_swarm or firmware.
  • matches(parameters, ...) now evaluates against serialized parameters, matching contains(parameters, ...) semantics.

Verification

  • cargo fmt --check
  • cargo test (199 tests)
  • cargo package --allow-dirty
  • cargo publish --dry-run --allow-dirty
  • mcp-publisher validate server.json

v3.10.1 — fix dropped user rules; ship Task* deny default

Choose a tag to compare

@jmcentire jmcentire released this 08 May 05:10

Fixed

  • load_merged_policy no longer drops user rules from rules.yaml when the system policy.yaml is missing or malformed. Previously this branch returned default_policy() and silently discarded every user rule, so on hosts without a per-host system policy both signet_test and real hook enforcement were no-op for user-installed rules. Missing/malformed system policy now falls back to the hardcoded baseline (self-protection + system defaults) and still merges user rules on top.
  • Three regression tests added covering missing system policy, missing system policy with self-protection preserved, and malformed system policy with user rules preserved.

Added

  • New locked default rule prefer_persistent_task_store: denies Anthropic's session-local Task* tool family (TaskCreate / TaskUpdate / TaskList / TaskGet / TaskOutput / TaskStop) and routes the agent to a persistent task store such as kindex's mcp__kindex__task_*. Ships locked so it cannot be silently overridden by an unlocking user rule.

v3.10.0

Choose a tag to compare

@jmcentire jmcentire released this 03 May 18:36

Added

  • Codex hook adapter support via --adapter codex and --adapter codex-permission.
  • Codex PreToolUse deny-only enforcement mapping and PermissionRequest allow/deny/defer mapping.
  • Codex hooks example at hooks/codex-hooks.json.
  • Integration tests for Codex hook response shapes.

Changed

  • Documentation and crate metadata now describe agent-agnostic support for Claude Code and Codex.

Verification

  • cargo test
  • cargo package --allow-dirty

v3.9.0 — Session-scoped preflights, vault protection, broad-rule advisory

Choose a tag to compare

@jmcentire jmcentire released this 04 Apr 01:58

What's New

Vault passphrase protection (locked rule)

AI agents can no longer run signet-eval setup or signet-eval unlock. This prevents an agent from setting a vault passphrase and locking the human out — which happened in practice.

Overly broad preflight constraint advisory

Submitting a preflight constraint with tool_pattern: ".*" and no conditions is now rejected with an explanation. An agent previously locked itself out by filing a universal constraint that matched every tool call including its own MCP tools. Use force: true to bypass if genuinely needed.

Session-scoped preflights (SIGNET_SESSION)

When the SIGNET_SESSION env var is set, preflights are scoped to that session. Each Claude Code instance sees only its own preflights plus global (session_id=NULL) ones. Deactivation, lock checks, and overrides are all session-aware. Backward compatible — unset SIGNET_SESSION means identical behavior to v3.8.0.

Files changed

  • src/policy.rs — new locked rule protect_vault_passphrase (8th self-protection rule)
  • src/vault.rs — session_id field on Preflight, DB migration, session-filtered queries
  • src/mcp_server.rs — broad-constraint advisory, session_id capture + display
  • src/main.rs — CLI preflight status shows session scope

Tests

138 tests passing (107 unit + 7 integration CLI + 24 integration hook)

v3.8.0

Choose a tag to compare

@jmcentire jmcentire released this 02 Apr 06:37

What's New

Added

  • validate --fix CLI flag — auto-fixes clampable issues (gate.within, ensure.timeout clamping) and removes broken unlocked rules; writes updated policy and re-signs if vault exists
  • validate --fix --dry-run — previews what --fix would change without writing to disk
  • status now shows complete enforcement state: global disable, disabled sessions, global pause with expiry, per-rule/per-session pauses with timestamps
  • status shows enforcement overrides even without a vault (file-based state is independent)

Fixed

  • enable (no flags) now clears both global disable AND all session disables in one invocation; previously required running twice if both existed
  • status no longer returns exit code 1 when vault is not set up — enforcement info is still useful without a vault