Skip to content

v0.6.1

Pre-release
Pre-release

Choose a tag to compare

@cryptojaz cryptojaz released this 19 Sep 03:50
· 21 commits to main since this release

The first evening with the hook installed produced 151 real decisions: 55% allow, 40% ask, 5% deny. That ask rate was unusable — and the audit log showed most of it was toolgate's own policy, not the model. Every change in 0.6 comes from that log (docs/usage-2026-09-19.md).

Fewer false positives

  • Input cap 6 000 → 20 000 chars. The old cap alone caused 62% of asks: ordinary source-file Writes and Edits on which the model had already said "all risks below 55%".
  • Task context is now the latest user prompt plus the two before it. In a working session the latest prompt is usually "yes" or "go ahead"; off_task was being judged against two words.
  • The transcript is read backwards in chunks until the prompts are found. 39% of calls had no task context because large tool results pushed the last prompt out of a fixed tail.
  • Wording: stopping or restarting your own dev server is not destruction (four of seven real denies were kill/pkill on a dev server). Writing code that would call an API is not exfiltration. "Ask me before X" is not a prohibition.
  • A reserved choice is a question, not a block: when the model is deny-level sure the task reserves a decision for you, a deny on a softenable axis becomes an ask.

Unattended modes

  • In bypassPermissions, auto, and dontAsk nobody answers a prompt — all 60 asks in the log were resolved without the user seeing one. An ask there is now a deny, with the reason fed to the model. unattended.ask: ask turns this off.

Diagnosability

  • Audit log records permission mode, whether task context was found and how long it was, input truncation, and the first line of the task. toolgate audit --stats reports them.

Evaluation (real Jev, recorded as-is in docs/)

  • Frozen held-out set 2: 20/20. Frozen held-out set 3: 20/20 (17/20 at 0.5). New development set 4, built from the real-usage failures: 9/12. Zero permissive errors across all 52 cases; every miss is stricter than desired.

npm install -g @riskaverse/toolgate@0.6.1 && toolgate init