Repository navigation
Releases
Releases · carlslin/dsh-skill-eval
v1.21.0 — TypeScript sources and per-issue copy
- The whole codebase is TypeScript (strict). Sources moved to
src/
(src/index.tshost entry,src/lib/*,src/scripts/*.mtsCLIs,
src/client/*.tsbrowser snippets,test/*.ts).npm run compileemits the
same runtime paths the package always shipped (index.js,lib/*.js,
scripts/*.mjs,client.js) — the tarball contents are unchanged.
npm run typecheckruns stricttscover the host, client and test
projects; eslint now covers.ts/.mtsvia typescript-eslint. - A git clone is not an installable plugin. Runtime files are build output
and are not in the checkout. Install from source withnpm install,npm pack,
thendsh plugin addof that tarball. Tests importsrc/, so the suite runs
the TypeScript you edit.npm run build:clientcompiles before it writes
client.js. The published tarball omitsscripts/build-client.mjs, which
imports thetypescriptdevDependency. - Every issue can be copied on its own. Each skill advice row and each MCP
server/tool finding has its own "复制修复指令 / Copy fix" button that writes a
self-contained Markdown repair task (title, evidence, concrete fix) — no more
copying the whole report to hand a single problem to an agent. Clipboard API
first,execCommandfallback preserved. npm testworks on Node 22.18 and 24 again. The suite command uses an
explicittest/**/*.test.tsglob because Node 22/24 treat a bare directory
argument as an entry module;release:checkandscripts/build.mtsshare it.- A fresh profile no longer fails acceptance.
acceptskips telemetry,
snapshot markdown and snapshot JSON until the first skill load, and only
prints the stale-generation hint for generation-shape failures;verify
skips per-turn boards and the latest snapshot until data exists. The
clean-profile release smoke can now pass without data. - Tests: 286 passing (
npm test).
v1.20.5 — atomic stale-lock takeover
1.20.5 — atomic stale-lock takeover
- A stale
events.jsonl.lockis now taken over atomically: the waiter renames the dead lock away before recreating it, so two processes that both saw the same crash leftover can no longer delete the winner's fresh lock and write concurrently. - The 10 s stale threshold and the 15 s acquisition timeout are unchanged.
- Regression test: two stores start against one stale lock, both records land, and no
.staleleftovers remain.
Tests: 280 passing (npm test).
v1.20.4 — review fixes and hardening
1.20.4 — review fixes and hardening
Review fixes
- MCP tool advice names the tool in Chinese copy (
gh/search). - Edit impact counts failed loads as failures (
degraded, notimproved). - The last step of a turn is flushed even when no skill window opened.
- The dead-load noise gate resamples the same root/evicted base as
abandonRate. loadErrorsis capped at five messages; the failed count is unchanged.- A cleared numeric setting is an error, not a silent zero; a saved
settings.jsonis validated before it is applied.
Telemetry hardening
events.jsonlis bounded at 8 MB. Rotation keeps dropped-prefixeditrecords (up to a quarter of the cap) ahead of the newest suffix, and the trimmed file is at most half the cap.- Appends serialize across processes through
events.jsonl.lock(a 10 s stale lock is a crash leftover), so two hosts sharing one data dir no longer lose records to a replaced inode. - New data directories are
0700, every file inside is0600, and an existing directory keeps the mode the user gave it. Leftover.tmpfiles cannot leak a stale mode. /data, the board and the config page return a fixed message on 500 and log the detail instead of echoing it.view:"dev"reports rejected saved settings (settingsErrors+ thesettings-invalidfinding).
Tests: 279 passing (npm test).
v1.20.3 — review hardening
1.20.3 — review hardening
Independent host/client/packaging review fixes, all with regression tests:
- Per-turn strip counts real tool errors (DSH
ToolChatData.root) and retries when its report cache predates the turn. - Settings writes merge inside the write queue (no lost fields);
windowTtlMs,reportOnTurnEnd,turnReportsandpromptHintnow apply live. - Failed
skillloads no longer open or supersede evidence windows. - Corrupt window/MCP records and invalid timestamps cannot crash reports or history.
- Release checks are falsifiable: canonical-route fails on restart skew, turn-boards cannot pass with zero boards, manual rows are INFO, and displayed weights always sum to 100.
- accept CLI reads the served dataDir and parses
--data-dircorrectly; the packaging doctor passes in an installed tarball and--releaseinspects its contents. - Versioned routes that answer 200 HTML fall back to canonical JSON for data and settings.
- Per-turn boards carry the settled turn; snapshots are atomic; analyzer/listener teardown is symmetric.
- Settings fence also blocks DNS-rebinding Hosts; 403/413/rebinding tests added.
- Dev tab renders settings file and warning count; settings labels are accessible; score colours follow configured thresholds.
Tests: 258 passing (npm test). doctor and doctor --release pass; verify-live is 7 checks + 2 manual.
v1.20.2 — MCP board and settings readability
Public release of dsh-skill-eval — skill execution telemetry, scoring and evidence-backed advice for DeepSeek Harness.
Highlights
- Board with skills / steps / runs / MCP / dev tabs, per-step trajectory scoring, per-turn result strip.
- 0–100 scores with 95% bootstrap intervals; overlapping intervals are reported as a non-significant rank.
- Advice carries evidence: every item names the number that triggered it and a concrete fix.
skill_eval_report/skill_eval_reviewagent tools and/skill-evalcommands.
In 1.20.2
- MCP tab rebuilt: latency meters with a configured threshold marker, a carded server table with grade chips (no more clipped columns), carded advice and failing-tool sections; a single empty card when nothing is recorded.
- Settings readability: percent units on weights, clearer grade-floor help, localized units, tighter dialog title, readable dark-mode headings.
- Dev tab reports real diagnostics: loaded vs on-disk version (restart-skew), claimed routes/tools, settings file, warning count.
- Config opens in-panel: the desktop shell denies
window.openforhttp://127.0.0.1, so the button now opens the settings form in a dialog.
Install
dsh plugin --profile desktop add dsh-skill-evalOr from source:
git clone https://github.com/carlslin/dsh-skill-eval.git
dsh plugin --profile desktop add "$PWD/dsh-skill-eval"Then refresh the GUI: the board is the Skill eval panel icon in the sidebar.