Skip to content

LANCET Nano v0.4.2

Pre-release
Pre-release

Choose a tag to compare

View all tags
@TannerMidd TannerMidd released this 29 Sep 04:40
· 5 commits to main since this release
v0.4.2
2b18ce8

LANCET Nano v0.4.2: a newly trained model with the highest Triage Score of any guard tested. It catches more risky Bash commands and stops fewer safe ones overall. 110M parameters, 111 MB INT8 ONNX, about 14 ms per command on a CPU, offline.

Model: Apache-2.0. Runtime: MIT.

What's new

  • Triage Score 82.7 (v0.4.1: 69.7). Every risky command asked about or blocked earns a point; the score shrinks when more than 10% of safe commands are stopped. Three benchmarks (5,500 commands), weighted by size.
  • Newly trained from the pinned CodeT5+ 220M base on a larger, more balanced mix (43,418 rows). New sources: the Shell Safety v2 training split (MIT) and project-authored contrast sets of agent-style, exec-style and shell-escape commands with safe look-alikes.
  • Far fewer interruptions on everyday commands: on the 513-command neutral set it stops 7.0% of safe commands (v0.4.1: 16.9%) and catches 91.0% of risky ones (66.5%).
  • More asking, less blocking: more of the risky commands it catches come back as review rather than risky.
  • Same architecture, size, runtime and speed as v0.4.1.

Results at the shipped thresholds

v0.4.2 v0.4.1
Triage Score 82.7 69.7
lancet-bench-1: risky caught / safe stopped 92.4% / 9.9% 91.7% / 9.4%
ShellRisk-Bench test: risky caught / safe stopped 60.1% / 2.7% 70.5% / 3.1%
Neutral set (513 commands): risky caught / safe stopped 91.0% / 7.0% 66.5% / 16.9%
lancet-bench-1: risky secrets caught (112) 78% 77%

Triage Score by release: v0.1.0 55.4 · v0.2.0 55.3 · v0.3.0 61.7 · v0.4.0 69.2 · v0.4.1 69.7 · v0.4.2 82.7.

Verify and run

  1. Check lancet-v0.4.2-nano-cpu-int8.zip against SHA256SUMS.txt.
  2. Extract it and run python verify_bundle.py --strict.
  3. Follow the bundle's README.
  • ONNX SHA-256: 1b6249c369ad390682d034fb9ee872dcecaa0c3418eb89218b6fa6111594a547
  • ZIP SHA-256: dfe70d0cd84142acb5551f9972557f4cddb5ea76477d7ed99fbbd070170f06f9

Inputs are inert text and are never executed. not_flagged is not execution authorization or a safety guarantee.

Credits and source licenses: NOTICE.txt and THIRD-PARTY-NOTICES.md in the bundle.

Assets 4
Loading