Repository navigation
Releases: xienda/dsh-jev-verify
Release list
v0.8.6
dsh-jev-verify 0.8.6
Fixes the startup crash reported in #1.
Root cause
@deepseek-ai/dsh-tools was declared as a normal dependency (0.1.5-rc.2) - the only plugin in the ecosystem that did. On a profile using nodeLinker: hoisted + hoistPattern: ["*"], pnpm materialised a real copy under <profile>/node_modules/@deepseek-ai/dsh-tools. The loader entry id: tools resolved to that copy instead of the host's built-in one; the copy could not import its own peers (Cannot find package '@deepseek-ai/cordis'), so the tools service never registered, every plugin with inject: ["tools"] stalled, and startup ended with:
DesktopHostFatalError: dsh: startup failed: 1 required plugin did not activate
Fix
@deepseek-ai/dsh-toolsmoved fromdependenciestopeerDependencies(^0.1.5-rc.2) and is markedoptional: trueinpeerDependenciesMeta, so no installer will ever materialise it - it cannot shadow the host.dependenciesnow carries only the leaf library@deepseek-ai/schemastery.- Runtime resolution is unchanged:
lib/index.js:28-39imports both host packages defensively (null on failure) and the host's own store still provides them. - New
test/manifest.test.mjsfails if any host runtime package reappears independencies- verified by reintroducing the bug and watching the test go red. Full suite: 38/38.
Upgrading from a broken install
Removing the plugin entry is not enough, because the hoisted copy outlives it. After upgrading to 0.8.6, delete <profile>/node_modules/@deepseek-ai/dsh-tools (or re-run pnpm install) and the host boots again. The same applies to the manually installed copy of @deepseek-ai/dsh-tools under any profile that used a hoisted linker.
Everything else is identical to 0.8.5: seven tools including jev_batch, a ledger that survives restarts, a remaining balance derived from your declared balance minus locally measured spend, per-turn auto-triage, and the live 27-question benchmark (96.3%).
v0.8.5
dsh-jev-verify 0.8.5
The ledger survives a restart, the remainder is honest, and a new tool does the paperwork.
0.8.5 answers a real session complaint ("the panel still shows nothing, and there is
no remaining balance") and turns the plugin from something you must remember to call
into something that does a whole batch of judgments in one shot.
1. The ledger survives a restart
The local ledger now persists to $DSH_HOME/jev-usage.json by default (quota.persist
defaults to true), so the cumulative total no longer resets with the host.
Two real bugs behind the old silent zero were found and fixed:
- the ledger path only honoured
DSH_HOME; the host process does not export it, so
defaultUsageFile()returnednulland persistence was silently disabled. It now
falls back to%USERPROFILE%\.dsh\jev-usage.json. save()ran before the firstload()(it is triggered by the firstrecord()of a new
process), so it wrote the still-empty in-memory ring over the real history file, and the
follow-up write was then swallowed by the 3 s debounce.save()now loads first, and
configure()only writes when it actually has data.
A restart round-trip probe (record 3 entries, build a new module, read back) returns
today=3 / all=3 and the same per-tool and per-kind breakdown.
2. Remaining is derived, never invented
TypeSafe exposes no balance endpoint: /v1/usage, /v1/quota, /v1/account,
/v1/account/balance, /v1/balance, /v1/credits, /v1/billing, /v1/me, /v1/limits
and a dozen more all return 404, and only /v1/models answers — with no balance headers.
So the panel derives the remainder instead:
remaining = quota.declaredBalanceUsd - locally measured spend since quota.balanceSince
Set those two fields in Settings → Plugins → Jev and the pill leads with
Jev · 剩余 $4.9997 · 今日 3 次. Without them it says plainly that it cannot be fetched
instead of printing a number.
3. jev_batch — one set of judgments over many texts
New module lib/batch.js, and a seventh tool:
- 1–20 items × 1–5 questions, one real API call per item, 4-way concurrency by
default (6 max) - returns a paste-ready markdown table, sortable by a judged dimension
(autosorts descending only when the single question is a score) - built-in
sentiment/priority/intent/pii/spampresets, or your own
questions in thejev_decisionshape - a failed row is reported as failed with the raw error — never guessed — and if every
row fails the tool throws the first error - every row is metered separately (
kind: "batch"), so N items really are N calls and N costs
Live end-to-end (bench/batch-e2e.mjs, real API, 3 + 2 items, 5 calls, ≈$0.0000744):
sentiment over three Chinese feedback lines sorted 2/4 (100%), 1.23/4, 0.93/4 in 708 ms
wall; then has_pii + urgency over two texts gave 100%/0.53 and 1%/0.01 in 261 ms.
4. Smaller honest fixes
- the composer pill re-reads every 20 s instead of 60 s
- the collapsed pill leads with the remainder once a balance is declared, and the
cumulative row is renamed when the ledger is on disk - tests are isolated from the real ledger (
test/_isolate.mjspointsDSH_HOMEat a temp
dir); 12 fixture lines a previous test run had appended to the real
~/.dsh/jev-roll.jsonlwere removed
Tests
npm test → 37/37 (~2.5 s): new test/batch.test.mjs (7 cases), new balance and
restart assertions in test/usage.test.mjs, pill/balance cases in test/client.test.mjs,
and the tool count moved to seven across boot/compat/present/toolview.
Scope
- Client half (pill, cards): refresh the page.
- Server half (persistence, remainder,
jev_batch): restart the host.
v0.8.4 - honest money-first panel, metered guard, auto-triage
dsh-jev-verify 0.8.4
Honest panel, metered guard, and a plugin that participates on its own.
0.8.4 answers three user-visible complaints from a real session: the panel implied
a quota that does not exist, the guard's own Jev calls never reached the ledger,
and the agent simply never called Jev.
1. No fabricated quota
TypeSafe is a metered API with no balance endpoint. If you never configured a
local budget, the old panel still printed "额度正常", drew a percentage bar against
∞ and showed "额度重置 13 小时 41 分后" — a countdown to a budget that did not exist.
- New
quota.budgetConfigureddrives every surface:jev_usagetext, the/jev
status page, the pill JSON route, the composer pill and the full dashboard. - Without a budget: no status verdict, no bar, no countdown; the text says
pay-as-you-go, nothing resets, only locally measured usage is reported. - With a local budget: the countdown returns, explicitly labelled
"本地预算重置 / 本地自设上限", and percentage bars become meaningful.
2. Money first
The collapsed pill now reads Jev · 今日 $0.000534 · 29 次; the popover leads with
today's cost and cumulative cost, and call counts are secondary rows.
3. The guard's Jev verdicts are metered
judgeRisky/judgeStall now return { noul, latencyMs, usage } (token usage comes
from body.usage, since requestSystemOne returns { body, latencyMs }) and report
through a new onJevCall event. index.js records them as kind: "judge",
tool: "auto-guard".
Before: jev_guard_status reported safety.jevCalls: 2 while jev_usage said
"今日调用 0 次". That gap was a real accounting bug, not a wording issue.
4. Auto-triage: the plugin judges the turn itself
New module lib/triage.js. On the first step of a turn the plugin asks Jev four
routing questions (intent, needs_judgment, judgment_kind, recommended_tool) and
injects the advice as a plugin-sourced user message, so the agent reads a
recommendation instead of having to remember to call Jev.
- at most once per (agent, turn) via
WeakMap;maxCallsPerSession200 stateclipped to 6000 chars,minChars12,timeoutMs4000createUserMessageis imported lazily from@deepseek-ai/dsh-llm; if it does not
resolve, the call is still metered but nothing is injected- fail-open on unknown host shape, missing key, timeout, abort or any API error
- each call emits exactly one
onTriageledger event (kind: "triage")
5. Harder guidance
The system-prompt section adds mandatory-trigger rules, a ban on bypassing Jev
"because I already know the answer", and the real cost magnitude
(~$0.00002–0.0001 per call; 25 questions cost the same as 1).
Tests
npm test → 30/30 (~2.5 s). New test/triage.test.mjs (5 groups), plus new
assertions in test/usage.test.mjs (no-budget branch), test/client.test.mjs
(money-first label, no-budget pill) and test/guard.test.mjs (onJevCall payload).
Scope
- Client half (composer pill): refresh the page.
- Server half (guard metering, auto-triage, guidance, status page): restart the
host.
v0.8.3 - /jev stops 404ing, plus a composer usage pill
dsh-jev-verify v0.8.3
Settings you save now actually reach the plugin.
What was wrong
Saving a value in Settings → Plugins → Jev reported success and changed nothing. The card sent the
field name straight to scope.set(field), and that call stores path: [field] — a single
segment. A field such as dashboard.basePath was therefore written into the settings document as one
literal key named dashboard.basePath, which no schema resolves, so the value was stored and then
ignored by every reader.
Found on a live host, not by inspection: after the 0.8.2 restart the settings document was read over the
settings/describe RPC and its user layer held two literal keys —
"autoGuard.maxJevCallsPerSession": 60 and "dashboard.basePath": "jev" — while the resolved document
still showed the schema defaults (50 and /jev). The card had lied politely for several releases.
The same read confirmed that the 0.8.1 fix is live: the decoded value has no empty-object paths and
carries apiKeyEnv: "TYPESAFE_API_KEY".
What changed
- Nested writes. The card builds
{ op: "set", path: [<segment>, …] }operations and commits them
in a single atomicscope.mutate(ops)call. Hosts that expose nomutatestill get the old chained
scope.setfallback, so nothing regresses on older harness builds. - Repair on open. A one-time mount effect rewrites documents written by <= 0.8.2 that contain literal
dotted keys: for each one it sets the nested path and unsets the literal key, all inside one atomic
mutation. AWeakSetmakes it run once per settings scope, and the whole effect is wrapped, so a
failed repair can never break the card. - Client-only. Refresh the page; no harness restart is required.
Verification
test/client.test.mjsPASS 7: path splitting, the generated operations (including numeric coercion),
exactly one atomic repair mutation with the expected operation order, no repeat on a second render, and
zero writes for a clean document.npm test→ 29/29 passing (2725.8 ms);node --check client/client.jsclean.- Live evidence (2026-10-07): the running host's
/jevanswered 200 with the status page and
/jev/api/usageanswered 200 with the pill payload; the served client bundle already carried 0.8.2,
confirming that a page refresh is enough for the client half.
Nothing else changed: six tools, the two guard tiers, the prompt section, the usage/quota accounting, the
/jev status page and pill, and every measured number from 0.7.5 / 0.8.0 / 0.8.1 / 0.8.2 stand as
published.
Upgrading
npm i dsh-jev-verify@0.8.3 and reload the page. The first time the card opens it repairs any literal
dotted keys left by an earlier version.
v0.8.2 - /jev stops 404ing, plus a composer usage pill
dsh-jev-verify v0.8.2
The board path stops 404ing, and the usage panel becomes a small pill by the composer.
What was wrong
Opening http://127.0.0.1:3080/jev returned the host's plain 404 page. Two independent mistakes:
- The routes were never mounted. The board only registered itself when
dashboard.enabled === true,
and that option defaults tofalse— so the path a user had configured (dashboard.basePath: jev)
existed nowhere. - The path was not normalized.
basePathwas used verbatim, so the valuejevregistered the literal
pathjev; a browser asks for/jev, which never matched. The trailing-slash form/jev/was not
registered either.
And the panel the board showed was the wrong shape for the request: a full standalone page is not a
lightweight usage readout next to the input box.
What changed
basePathis normalized (normalizeBasePath(): trim, add a leading slash, trim trailing slashes, fall
back to/jevfor an empty value or a bare slash) and bothbaseandbase + "/"are registered./jevnever 404s. Withdashboard.enabled: falsethe path now answers 200 with a server-rendered,
JS-free status page (today's calls, today's cost, this plugin instance, today's tokens, median latency,
quota state) that points at the composer pill and explains how to turn the full board on./jev/api/usagealways mounts — a small read-only JSON projection of the same locally measured
snapshotjev_usagereturns, with an honest{ok:false, error}body (still HTTP 200) when the snapshot
fails. It does not wait fordashboard.enabled.- A composer usage pill in the
conversation.input.rightslot (beside the submit button), modeled on
dsh-opencode-go:Jev · 今日 29 次 · $0.000534with a warn/danger tone dot, expanding into a 270 px
popover — today's calls and cost against their limits, this instance's calls, today's tokens, median and
p95 latency, guard denials, the top three tools, time to reset, the last refresh and a manual refresh.
It polls every 60 s and on tab focus, falls back to the full board API when the pill route is absent, and
degrades toJev · 额度不可用carrying the exact error. It never throws, and it reports local measurements
only — never a vendor balance.
Verification
test/dashboard.test.mjs(rewritten, 10 assertions): the exact route table with the board on
(/jev,/jev/,/jev/api/usage,/jev/api,/jev/api/try), with it off (three routes,/jeva 200
status page), and on a custombasePath: "/custom/"(six routes, still the full board); the normalizer on
six inputs including/and///(which fall back to/jevso the root is never hijacked); the pill
payload's fields with norollkey; and a throwing snapshot still yielding 200 plusok:false.test/boot.test.mjs: five routes registered, and withdashboard: {enabled:false}the/jevand
/jev/api/usageroutes exist while/jev/api/trydoes not.test/client.test.mjsPASS 6: the pill payload's three input shapes and three junk inputs, four label
states, five tone states, both popover states by text, the slot registration (id,order,name) and
the rendered fallback label.npm test→ 29/29 passing (2470 ms).
Nothing else changed: six tools, the two guard tiers, the prompt section, the settings card fix, the
usage/quota accounting and every measured number from 0.7.5 / 0.8.0 / 0.8.1 are exactly as published.
Upgrading
npm i dsh-jev-verify@0.8.2. The server half (routes, normalizer, status page) needs a harness restart; the
client half (the pill) only needs a page refresh — the host's client-hmr row stat-polls client bundles and
re-hashes them within 500 ms.
v0.8.1 - a configured key no longer reads as unconfigured (settings card fix)
dsh-jev-verify v0.8.1
The Settings > Plugins card now tells the truth: a configured key is never reported as "unconfigured".
The bug
Symptom: ~/.dsh/.env holds TYPESAFE_API_KEY, and the in-chat jev_overview reports
API Key 已配置, but Settings > Plugins > Jev showed a 未配置 (unconfigured) badge with every
field blank — as if nothing had ever been saved. Saving from the card looked like it did nothing.
Root cause, located layer by layer on a live host (dsh 0.1.5-rc.2):
- The settings service publishes the resolved value verbatim
(@deepseek-ai/dsh-settings/lib/index.js:364value: registration.resolved); nothing simplifies it. - The schema this plugin registered marked its GUI fields with the
volatilemarker. The host resolves
@deepseek-ai/schemasteryat 3.18.4, whose resolver (src/index.ts:769) turns every marked field
into a live reference object:schema.meta.volatile ? createVolatile(...) : default. - So the value on the wire was
{"enabled":{},"apiKey":{},"apiKeyEnv":{},"quota":{"enabled":{},…}}. - The GUI carries its own older, vendored schemastery (3.18.2) with no
volatilesupport at all, so
decode()(dsh-client-ui-settings/lib/client.js:1107-1117) failed its validation — the real error was
$.enabled expected boolean but got [object Object]— returnedundefined, and the draft stayed empty. - The card therefore read
value === {}and had no key to show.
The plugin's own config reads were never affected: normalizeConfig unwraps those references, which is
why judgments, the guard and jev_usage kept working the whole time. Only the human-facing card was blind.
The fix
- One field definition, two schemas.
buildConfig()builds bothConfig(entry form; keeps the
volatile marker for hosts that understand it) andSettingsConfig(plain — the one registered with the
settings service). The 23 fields cannot drift apart. - The card no longer depends on a single decode path. If the decoded value comes back empty it reads the
rawbase/userlayers the service already sends, shows an explicit note that the server value could
not be decoded (schema-generation skew), and stays fully editable and savable. - A credential reference is not the same as "configured". The badge has three honest states: a literal
key that was saved,环境变量 <NAME>(a reference, unverified), or none.
Verification
Wire probe against the deployed module (_wire_probe.mjs, 2026-10-06):
| schema | empty-object fields on the wire | apiKeyEnv |
GUI decode |
|---|---|---|---|
Config (entry form only) |
8 + every quota.* |
{} |
FAILS — $.enabled expected boolean but got [object Object] |
SettingsConfig (registered) |
none | "TYPESAFE_API_KEY" |
passes |
test/config.test.mjs— the registered schema resolves to plain, JSON-safe values; a save round-trip
stays plain.test/compat.test.mjsPASS 1b —volatilePaths(SettingsConfig)must be empty; wire probe asserts
enabled === trueandapiKeyEnv === "TYPESAFE_API_KEY".test/client.test.mjsPASS 4 — a mini-renderer drives the real card against a snapshot whose decoded
value is empty; the badge text must be环境变量 TYPESAFE_API_KEYand the tree must never contain
未配置.npm test→ 29/29 passing (2473 ms).
Nothing else changed: six tools, the guard tiers, the prompt section, the usage/quota panel and every
measured number from 0.7.5/0.8.0 are exactly as published.
Upgrading
npm i dsh-jev-verify@0.8.1. The server half (the registered schema) needs a harness restart; the client
half only needs a page refresh.
v0.8.0 — usage/quota panel (jev_usage)
dsh-jev-verify v0.8.0
Jev usage, measured — and an honest quota panel.
What's new
jev_usage: a usage/quota panel you can call
Every jev_decision, jev_choose, jev_verify and guard event is now recorded locally with its
measured input/output tokens, estimated cost, latency, question-type mix and tool name. jev_usage
turns that into:
- rolling windows today / 7 days / 30 days / all / session;
- calls, input/output tokens, estimated cost, latency median and p95, per-question-type and per-tool mix;
- a projected daily burn rate — "at this pace the daily call limit is reached in N h";
- a per-day sparkline over the retained history;
- an optional hard stop (
quota.enforce): a call that would cross a configured budget is refused
before the API request, with a message naming the budget. Display-only by default.
The honest boundary, stated everywhere
TypeSafe publishes no balance or quota endpoint. Reproduced 2026-10-05: GET /v1/usage,
/v1/quota, /v1/account, /v1/me, /v1/balance, /v1/credits, /v1/limits, /v1/billing,
/v1/subscription, /v1/plan, /v1/user, /v1/health and /v1/ all return 404; only
/v1/models answers (jev-latest, jev-preview). A panel that showed a number from nowhere would be
exactly the kind of surface this plugin refuses to ship, so the panel reports only what this machine
measured, against limits you set: quota.dailyCallLimit, quota.dailyCostLimitUsd,
quota.sessionCallLimit and quota.warnAtPercent. The boundary is printed in the tool description,
the text result, the in-chat card and the README.
Four surfaces
- In the conversation — a
usagetool view: status / threshold / reset-countdown chips, eight stats,
three budget meters, the daily sparkline, the per-tool breakdown and the boundary note at the bottom. jev_overview— a usage chip row (status, today's calls against the limit, today's cost, whether
the hard stop is armed).- The optional
/jevboard (dashboard.enabled: true) — a quota section with bars, projections and a
30-day trend. - Settings > Plugins — a new "usage/quota" group for every option, with numeric validation.
Persistence
In-memory by default (a 400-sample ring). Set quota.persist: true to keep up to quota.historyDays
(default 30, range 1–365) days of daily history in $DSH_HOME/jev-usage.json — written atomically
(tmp + rename) and fail-open: a panel problem can never break a judgment call.
Also in this release
jev_verifynow records its 27 calls, tokens and cost in the ledger instead of only the accuracy text.- Cost accounting uses one injected constant (
$0.042per million input tokens, output free) everywhere,
including the dashboard playground; no hard-coded copy remains. - Six tools, all registered and covered by boot/compat tests:
jev_decision,jev_choose,jev_usage,
jev_overview,jev_guard_status,jev_verify.
Verification
npm test→ 28/28 passing (2,540 ms). Newtest/usage.test.mjscovers the empty state and its
honest note, token-to-cost accounting, a 27-call batch (guard events counted separately, not as calls),
quota status and all three hard-stop paths, persistence with day-boundary pruning, reload, reset, and
every formatter.- Cost constant cross-checked against real measurements: 8,696 input tokens × $0.042 / 1e6 =
$0.000365232 ≈ $0.000365 — the same figure the 0.7.5 live report published. The panel derives cost
from measured tokens with that constant; it does not guess. - Live benchmark (2026-10-04, concurrency 6): 96.3% (26/27), median 282 ms, p95 1468 ms, 8,696 input +
822 output tokens ≈ $0.000365, wall 3999 ms versus 10.07 s serial (2.52×); report
bench/results/2026-10-04T12-40-07-421Z.json. - The guard audit from 0.7.5 still holds: two deterministic tiers (7 hard, position-aware + 3 soft), a
Jev backstop,onSafetyDeny/onLoopAdvisoryreported with the tool name, fail-open on Jev failure.
Upgrade notes
- Server half needs one harness restart (the tool, the settings group and the ledger live in the
server plugin). The client half only needs a page refresh. - Fully backwards compatible: all new options have defaults (
quota.enabled: true,quota.enforce: false,
quota.persist: false), so an existing config keeps working unchanged and the panel starts out
display-only. - Tier-9 pipeline patch: npm
dsh-jev-verify@0.8.0.
Links
- npm: https://www.npmjs.com/package/dsh-jev-verify
- Repository: https://github.com/xienda/dsh-jev-verify
- Verification report: docs/verification.md
v0.7.5 — honest ledger, two-tier position-aware guard, parallel jev_verify
v0.7.5 — the guard stops over-blocking, the ledger stops lying, jev_verify gets 2.5× faster
Release date: 2026-10-04 · npm: dsh-jev-verify@0.7.5 · commit: 57ccb0e · full diff: 6a91e7d...57ccb0e
A three-way audit of 0.7.4 (server, client, docs/packaging) produced 24 findings. This release fixes all of them. Three were more than cosmetic: the deterministic guard both over-blocked harmless text and under-detected real command variants, the dashboard ledger recorded nothing at all unless the dashboard page was enabled, and jev_verify ran its 27 live calls strictly serially at ~10.1 s — close to its own host timeout.
1. The auto-guard: two deterministic tiers, and it now reads position
Before, all patterns matched anywhere in the argument text, case-insensitively. Writing about a dangerous command — in a README, in a test, in the plugin's own guard source, in a commit message, even in these release notes — was blocked exactly like running it. At the same time the root-recursive-delete matcher required the target to follow the flags immediately, so extra flags in between, or the --flag=value form, slipped past it entirely and fell through to the shared risky-verb branch.
Now:
- 7 hard rules fire only at an executable position — the start of the text, or after a separator — and only when the tokens before the match are at most four wrappers / env assignments / flag words. A hard-rule match anywhere else is demoted to the soft tier and its pattern is reported with a
(quoted/described)suffix, so the audit trail still says which rule matched. - The root-recursive-delete matcher walks tokens instead of using a word regex: flags in any order, extra flags,
--flag=value, and separator-delimited commands are all handled; the first non-flag token after the flags is the target. Recursive deletes aimed at a build directory or a temp path are no longer hard blocks — they go to Jev as a soft signal — while a target that is the filesystem root, a home directory, a drive root or the current directory stays a hard block. - 3 soft rules (taking the host down, git-history rewrite short of a force push, always-true conditional on a delete/update) are always handed to Jev against
denyThreshold. - Totals are unchanged: 7 hard + 3 soft = 10 rules.
2. The guard now reports what it does
onSafetyDeny fires for both tiers and carries the tool name; the Jev tier adds the confidence. The loop check fires onLoopAdvisory with the tool name. Before, a single event existed for one tier only, which made two consumers dead code: the dashboard's Jev-verdict branch and its advisory counter always saw zero.
3. jev_overview no longer reports zeros
The ledger's record() returned early unless the dashboard routes had been mounted — and routes mount only when dashboard.enabled === true, which defaults to false. Result: every jev_overview call reported 0 calls, 0 tokens, $0, and $DSH_HOME/jev-roll.jsonl was never written, contradicting both the README and the settings text.
Fix: the in-memory ring always records; only the HTTP route and the JSONL append are gated by dashboard.enabled. Cost accounting uses the injected per-MTok constant instead of a hardcoded literal.
4. jev_verify runs six calls at a time
runCase + batches of Promise.all with VERIFY_CONCURRENCY = 6, results accumulated in case order. Measured wall clock 3.999 s versus 10.07 s serial (2.52×), comfortably inside the host timeout. Accuracy is unchanged at 96.3% (26/27).
5. Better errors, more honest audit
missingKeyError(tool)names the tool that was actually called. It used to hardcodejev_decision, sojev_chooseand the dashboard playground told users to check the wrong key.TYPESAFE_BASE_URL/TYPESAFE_MODELnow really override the schema default. The config default was read first, so a user pointing the plugin at a self-hosted endpoint was silently sent to the public API.- The in-session verdict cache is capped at 200 entries (
MAX_VERDICTS); a long session can no longer grow it without bound. jev_verifyreports the fullmislabeledlist plusmislabeledHighConfidence. Previously, one high-confidence mislabel replaced the whole array, so low-confidence misses disappeared from the report.safe(tag, fn)accumulates registration failures, andjev_guard_statusexposes them underregistrations, so a silently swalloweddefineToolfailure can no longer ship unnoticed.
6. Client fixes
- noul confidence now reports the probability of the answered side: a "no" at p 0.02 used to render "置信度 2%" with warning colour, while the overview card showed the same value as 2% — the two surfaces contradicted each other.
jev_chooseviews are auditable: the state text and every candidate's text are kept, so a ranking table can be read without re-running the call.- Overview cards show
avgLatencyMsandtypeCounts, which the server already sent and the card dropped. - Dashboard page shows an error surface instead of staying on "加载中…" forever when a fetch fails.
- Credential badge is tri-state: an env-var name alone reads "read from environment (not verified)" and never "configured".
- Removed the dead CSS variable (highlight and chip fill work again in dark themes); the running chip only claims "0 questions" where that is meaningful.
7. Packaging and docs
test/functional.mjs→test/functional.test.mjs, so the real HTTP-contract test actually runs innpm test(27 tests, was 26).bench/choose-e2e.mjsno longer hardcodes one machine's home directory: it readsTYPESAFE_API_KEYfrom the environment, then falls back to the DSH home.bench/bench.mjsgained--concurrency(default 6) and reports it.- README p95 band corrected to the measured tail (825–1468 ms; single-run peaks reach ~1.5 s and a congested network ~5 s), test count corrected,
engines.dshdocumented as informational only (npm does not enforce it), and the rule split documented as 7 hard / 3 soft / 10 total. package-lock.jsonbrought to 0.7.5.
Verified
npm test: 27/27 green (12 files, 2559 ms), including 9 guard suites that assert position-aware tiering, flag-order independence, both event payloads with the tool name, and the 200-entry cache cap.- Live benchmark (2026-10-04, concurrency 6): 96.3% (26/27), median 282 ms, p95 1468 ms, min 231, max 1471, wall 3999 ms, 8696 input + 822 output tokens ≈ $0.000365; report
bench/results/2026-10-04T12-40-07-421Z.json. The single mislabel is unchanged (severity-low, 0.01 vs expected 0). - Live
jev_chooseE2E (2026-10-04): recommended "verify, then publish" fit 1.89/3 · risk 22% · composite 49% · confidence 42%; the reckless option fit 0.1/3 · risk 90%; 933 ms, $0.0000674 (1605 in + 99 out), modeljev-1.13.0. - The guard rewrite is its own evidence: the old deterministic tier blocked this release's source, its commit message and these notes while they were being written; the new tier lets the same text through while still blocking the same operations at executable position.
Install / upgrade
dsh plugin --profile web add dsh-jev-verify
Afterwards, cycle the host process: plugin code is loaded at start-up, so a running host keeps the previous version in memory.
Honest caveats
jev_overviewcounts from the moment the plugin instance started; it is not a persistent history unlessdashboard.enabledis on (which is when the JSONL file is written).- Latency is network-dependent. The p95 above is one measured run; a congested network can reach ~5 s, which is why
DEFAULT_TIMEOUT_MSis 15000. - The guard is a fail-open backstop, not a sandbox: if Jev is unreachable or the per-session budget is exhausted, suspicious commands pass. Hard rules stay in force because they are local and free.
- Jev returns verdicts and calibrated probabilities, never explanations. Any reason shown in a tool view is written by the calling model, not by Jev.
中文摘要
三路审计(服务端 / 客户端 / 打包文档)在 0.7.4 上共提 24 条问题,本版全部修复。三处要害:确定性护栏既误拦又漏检——旧实现按整段文本做大小写不敏感匹配,写在文档、测试、插件自身源码、提交信息甚至本发布说明里的危险命令都会被硬拦,而对根目录递归删除的识别要求 flag 后紧跟目标,额外 flag 或 --flag=value 写法直接漏过;看板账本何时都记 0——record() 在路由未挂载时直接返回,而路由只在 dashboard.enabled 为真时挂载(默认 false),于是 jev_overview 恒报 0 调用 0 token $0,JSONL 永不落盘;jev_verify 串行 27 次调用墙钟 10.07 s,逼近宿主超时。
本版改动:护栏改为硬/软两层 + 位置感知(7 硬 + 3 软 = 10 条不变;硬规则只在可执行位置生效,正文/引号内降级为软并标注 (quoted/described);根目录删除改为逐 token 解析,flag 乱序、额外 flag、= 取值与分隔符之后均能识别;对普通相对目录的递归删除降为交给 Jev),两层都上报 onSafetyDeny(带工具名,Jev 层带 confidence)、循环检测上报 onLoopAdvisory(带工具名);账本内存 ring 永远记录、仅 HTTP 路由与 JSONL 落盘受 dashboard.enabled 控制,成本常量改为注入;jev_verify 以 6 路并发运行(墙钟 3999 ms,串行 10.07 s 的 2.52×),精度仍是 96.3%(26/27);missingKeyError 按工具名报错、TYPESAFE_BASE_URL/TYPESAFE_MODEL 覆盖真正生效、判定缓存上限 200、mislabeled 全量并新增 mislabeledHighConfidence、注册失败累积到 jev_guard_status.registrations;客户端修 noul 置信度、jev_choose 可审计、概览卡补 avgLatencyMs/typeCounts、看板错误面、凭据徽章三态、死 CSS 变量;打包把 functional 测试改名以真正进入 npm test(27 项)、bench 去掉硬编码家目录并支持 --concurrency、README 延迟尾部分位与测试计数诚实化、lockfile 升到 0.7.5。
实测:npm test 27/27;真实基准 2026-10-04(并发 6)96.3%(26/27),中位 282 ms、p95 1468 ms、墙钟 3999 ms、约 $0.000365;真实 jev_choose E2E 推荐项契合 1.89/3 · 风险 22% · 933 ms · $0.0000674。诚实边界:账本自插件实例启动起算;护栏是 fail-open 兜底而非沙箱;Jev 只给判定与概率,不给解释。
v0.7.4 — the guidance section really registers (and jev_choose actually works)
v0.7.4 — the guidance section really registers (and jev_choose actually works)
Release date: 2026-10-01 · npm: dsh-jev-verify@0.7.4 · commit: 64ee834 · full diff: 9dd64a6...64ee834
This release fixes two silent registration failures. Both shipped code that looked right in tests and did nothing in a real host; both now have regression tests that fail when the real host contract is violated.
1. The system-prompt guidance section now exists — this is what makes the harness call Jev on its own
Until 0.7.3 the plugin asked the host for its prompt slot via getSectionOrder("TOOL_JEV"). That slot does not exist in the current DSH generation (SECTION_ORDERS ends at TOOL_REPORT = 2900, then jumps to TOOLS_SDK = 5000), so the lookup returned undefined — and SystemPrompt.section() rejects a non-finite order:
TypeError: prompt section "<name>" order must be a finite number
The throw was swallowed by the same per-surface quarantine that protects the profile, so the section silently never existed and the model only ever saw the tool descriptions. That is why the harness did not reach for Jev on its own.
Fix: resolve the order as getSectionOrder("TOOL_JEV") → guidance.order → DEFAULT_GUIDANCE_ORDER = 3000, register the section as tool:jev, leave the text empty when jev_decision is not registered, and log the outcome:
[dsh-jev-verify] system prompt: guidance registered | section tool:jev | order 3000 | chars 838
The section is written as trigger rules (tool + situation + boundary), not marketing prose: what each of the five tools is for, when to use it, when not to (no text generation, no long-chain reasoning), the confidence < 0.6 → re-check with the main model rule, and the honest "Jev scores, it does not explain" contract. Measured length: 838 characters with the auto-guard on, 717 with it off.
New guidance config group — enabled / order / extra — all volatile so they are editable from the settings card without a restart.
2. jev_choose was not callable in any host in 0.7.3 — now it is
The tool declared minItems/maxItems on its options array, keywords the host value-schema DSL rejects:
JsonSchemaError: unsupported JSON schema: parameters.options.minItems is not supported by the value schema DSL
defineTool threw, the per-tool quarantine swallowed the throw, and the boot test only asserted four tools — so 0.7.3 published a headline feature that no host could call. Dropped the keywords; the real constraints (2–10 options, ≤800 characters per option, ≤2000 characters of context) are now enforced at runtime by validateOptions in lib/counsel.js, and test/boot.test.mjs asserts all five tools register through the host's real defineTool schema compiler.
3. Honesty and robustness
choosePresentation(value)never throws and is JSON-reversible —presentationMetais projected while a call is running or failed, andundefinedkeys vanish in a JSON round-trip, so only defined keys are copied.formatChoose(value)degrades tojev_choose | 尚无结果(仍在运行或已失败)instead of throwing on a non-object.test/boot.test.mjs's fake context now mirrors the real host contract (section()rejects a non-finite order,getSectionOrder()only knowsTOOL_GOAL), so the contract that failed silently is now covered by a failing test.- Tool descriptions for all five tools were rewritten to say exactly what each one returns and when it must not be used.
Verified
npm test: 26/26 green.- Live API E2E (
bench/choose-e2e.mjs, key read from~/.dsh/.env, never printed): recommended 「先验证后发布」 fit 1.93/3 · risk 23% · composite 50% · confidence 41%; trap option 「直接发布」 fit 0.11/3 · risk 90%; 「只发 npm」 1.03/3 · risk 56%. Total 895 ms, $0.0000674 (1605 in + 99 out), modeljev-1.13.0. This is exactly the code path 0.7.3 never ran. - Benchmark (unchanged since 0.7.2): 96.3% (26/27) on the built-in 27-question set against the live API, measured 2026-09-21 and re-measured 2026-09-28; median 283–484 ms on
jev-latest; $0.000365 per full run. The single mislabel isseverity-low(score 0 → 0.01). - Boot-log proof this release is live:
[dsh-jev-verify] settings namespace registered: jev-verify | schema: present | scope: object.
What the plugin gives a DSH agent
- Five verdict tools:
jev_decision(1–25 parallel atomic judgments: choice / score / noul),jev_choose(rank 2–10 candidate approaches),jev_overviewandjev_guard_status(read-only audit),jev_verify(27-question self-check against the live API). - In-chat tool views — structured panels for every call, with latency, cost, model, threshold and expandable state preview.
- A full settings card (Settings → Plugins → Plugin configuration → Jev) covering the auto-guard, guidance, dashboard and budget options.
- An auto-guard on every shell-class call (
bash/pwsh/run_code/terminal): deterministic hard rules first, then a Jev risk judgment againstdenyThreshold; stall/loop detection only advises and never blocks. Jev failure is fail-open. - A local
/jevdashboard (opt-in,dashboard.enabled).
Install / upgrade
dsh plugin --profile web add dsh-jev-verify
Restart the host afterwards. A correct boot prints both lines quoted above; the guidance line is the one that proves the harness can now reach for Jev by itself.
Honest caveats
- Skip 0.7.3 for
jev_choose: the tool existed in the package but registered in no host. - The guidance section is the mechanism behind proactive Jev use; with it absent the model sees only tool descriptions (this was the 0.7.3 and earlier behaviour).
- Real API only. Without
TYPESAFE_API_KEYevery tool returns an explicit error; results are never fabricated, and there is no mock fallback.
中文摘要
v0.7.4 修掉两个「静默注册失败」——它们在测试里看起来是对的,在真实宿主里却什么也没发生,现在都有会真失败的回归测试兜底。
- 引导段真的注册上了(这才是 harness 会主动调用 Jev 的原因)。0.7.3 用
getSectionOrder("TOOL_JEV")取槽位,而这个 DSH 代际的槽位表里根本没有TOOL_JEV→ 返回undefined→SystemPrompt.section()抛TypeError: prompt section "<name>" order must be a finite number→ 异常被隔离逻辑吞掉 → 引导段从未存在,模型只看得到工具描述。修复:order 解析链getSectionOrder("TOOL_JEV")→guidance.order→DEFAULT_GUIDANCE_ORDER = 3000,段名tool:jev,成功日志[dsh-jev-verify] system prompt: guidance registered | section tool:jev | order 3000 | chars 838。该段写的是触发规则(每个工具何时用、何时不要用、置信度 <0.6 回主模型复核、Jev 只给判定不写理由),不是宣传语。新增guidance配置组(enabled / order / extra)。 jev_choose在 0.7.3 的任何宿主里都调不到。它的options参数声明了minItems/maxItems,被宿主值模式 DSL 拒绝(JsonSchemaError: unsupported JSON schema: parameters.options.minItems is not supported by the value schema DSL),defineTool抛错被单工具隔离吞掉,而 boot 测试只断言 4 个工具 —— 于是 0.7.3 发布了一个没人能调的主打功能。现已删掉这两个关键字,2–10 个候选 / 每项 ≤800 字符 / context ≤2000 字符由lib/counsel.js的validateOptions在运行时保证。- 诚实性与健壮性:
choosePresentation永不抛且 JSON 可逆;formatChoose对非对象降级为jev_choose | 尚无结果(仍在运行或已失败);boot 测试的 fakeCtx 改为镜像真实宿主契约(非有限 order 会抛、只认TOOL_GOAL);五个工具的描述全部改写为「返回什么、什么时候不要用」。
验证:npm test 26/26 全绿;真实 API E2E 中「先验证后发布」契合 1.93/3 · 风险 23% · 综合 50%,陷阱项「直接发布」0.11/3 · 风险 90%,总 895 ms、$0.0000674(1605 in + 99 out)、模型 jev-1.13.0;内置 27 题基准 96.3%(26/27),2026-09-21 实测、2026-09-28 复测,jev-latest 中位 283–484 ms,单轮 $0.000365。
安装/升级:dsh plugin --profile web add dsh-jev-verify,然后重启宿主。请跳过 0.7.3 的 jev_choose(该版本该工具在任何宿主都无法注册)。本插件只走真实 API,没有 key 时明确报错,绝不伪造结果。