Skip to content

v0.6.4 — tuned on real usage and adversarial review

Pre-release
Pre-release

Choose a tag to compare

@cryptojaz cryptojaz released this 19 Sep 05:05
· 13 commits to main since this release

The 0.6 line rebuilt toolgate's decisioning around two things it didn't have before: real-usage data and an adversarial evaluation.

From 151 real decisions (docs/usage-2026-09-19.md): input cap 6k→20k, task context is now the last three prompts read backward through the transcript, and the destructive/exfiltration wording stopped flagging "stop my dev server" and "write code that calls an API." In unattended permission modes an ask becomes a deny (auto mode is attended and unchanged).

From adversarial review (challenge set 5, 24 held-out cases — injection, forged approval, encoded targets, reservation transfer):

  • Reserved-choice softening requires task context and low off_task.
  • It never reduces off_task itself.
  • Secret-exposure and prohibition verdicts stay unsoftenable (keeping their scored verdict — an unsoftenable axis preserves its level, it does not force deny).

Regression suites (real Jev): set 2 20/20, set 3 20/20, set 5 23/24 — zero permissive errors, zero dangerous allows across all 64 cases. The one set-5 miss is a bounded, stricter-than-desired false positive (a single-quoted literal read as a substitution), documented and not chased with threshold changes.

Reviews by ChatGPT (GPT-6 Astra), who authored the challenge sets. npm i -g @riskaverse/toolgate