Skip to content

Releases: samirliu/dsh-goalloop-agentteams

v0.9.2 - Windows support (explicit shell resolution)

Choose a tag to compare

@samirliu samirliu released this 19 Oct 03:11
ea48118

Windows support: the gate no longer assumes POSIX.

Install:

dsh plugin --profile <profile> add https://github.com/samirliu/dsh-goalloop-agentteams/releases/download/v0.9.2/dsh-goalloop-agentteams-0.9.2.tgz
  • Shell resolution: POSIX /bin/sh -c, Windows %COMSPEC% /d /s /c (windowsVerbatimArguments); override with plugin config.shell, GOAL_GATE_SHELL or bin/gate.mjs --shell (sh/bash/pwsh/powershell/cmd).
  • Every gate result reports the resolved shell; a missing shell binary is BROKEN with an actionable reason; preflight adds posix-syntax-on-cmd; ETIMEDOUT -> rc=124.
  • Windows runtime fixes: app.asar defineTool fallback via pathToFileURL; atomic ledger-lock publish + writeState rename retries; parseSkillFile CRLF normalisation (CRLF checkouts silently dropped the runtime skills).
  • Tests are platform-neutral: 10/10 suite files in both the default cmd.exe gate and a Git Bash sh gate; 0/25 failures under 2-process concurrency; 10/10 on a real CRLF git checkout.
  • POSIX behaviour unchanged (same contracts, same verdicts, same deny shapes).

Full write-up: #5

v0.6.2 — 时间面与哑巴失败全部点名 (defects #13-#16)

Choose a tag to compare

@samirliu samirliu released this 11 Oct 13:34

judged unverifiable 全部带 why(无绑定/摘要过期/无判官/判官127)+预检黄牌; state-error 不烧轮; lastRunAt/gate ms/loop-idle 检测; 同结论重评点名(复查走只读 /goal-gate)。6 suites 全绿。

v0.6.1 — transport hardening: tool returns survive the lossless-JSON boundary

Choose a tag to compare

@samirliu samirliu released this 10 Oct 11:11

Two host-transport defects that silently killed working gate calls:

#11 — schema drift. goal_loop_at returned preflight without declaring it in the output schema; harness output validation rejects the whole call. Schema now declares it, with a live-return key-alignment regression ([10b]).

#12 — lossless-JSON. The host validates every tool return through a lossless-JSON check (nested undefined / NaN / ±Infinity / -0 all fail). runGate legitimately leaves why/stderr/value/bestScore/neverFailed/preflight unset — and every such call was reported as 'value is not lossless JSON' after its round had already been recorded: work done, verdict lost. A recursive jsonSafe() now sanitizes every return path. Mutation-tested: remove the sanitizer and the regression test fails immediately.

Also: bin/gate.mjs (the CLI loop driver that recovers sessions rely on when tools are unreachable) was missing from files[] and shipped in no release until now.

v0.6.0 — stress campaign: ten defects found in real acceptance work and fixed

Choose a tag to compare

@samirliu samirliu released this 10 Oct 10:04

A ten-scenario cross-domain stress campaign (data pipeline / HTTP service / semver solver / doc generator / CLI filter / performance / simulation / security / i18n-a11y / multi-member toolkit) hunted defects by running real acceptance loops. All ten scenarios ended GO 1.0; ten plugin defects were found, fixed and locked by regression tests:

  1. loop-scoped rounds (multi-experiment budgets no longer collide) and stamp-scoped never-failed
  2. preflight warns numeric-without-metric (last-number fallback is fragile)
  3. per-AC timeout field (long-running acceptance is no longer silently killed)
  4. extractMetric takes the first defined capture group (alternation regexes)
  5. unparsable AC lines refuse evaluation (silent drop forbidden)
  6. expected: exit=N (error-path assertions are first-class)
  7. tolerance: relative noise band (benchmarks are noisy; no more false reds)
  8. stderr diagnostics captured in results (fail-loud)
  9. ANSI color codes stripped before metric matching
  10. delta-saturation diagnosis (ceiling-bounded metrics get an actionable hint)

Plus the quality-face work of v0.4/v0.5: deliverable declaration, mandatory judgment evidence binding, quality score into the metric pipeline. Full details: docs/defect-ledger.md. Verified live, including the in-host pre-execute interceptor.

v0.5.0 — quality score enters the metric pipeline (passing ≠ excellent)

Choose a tag to compare

@samirliu samirliu released this 10 Oct 07:53

A real case scored GO 1.0 while its deliverable was visibly mediocre — the eval measured correctness but never craft quality, and the metric face only held the floor (no-regression) instead of pushing up.

Quality is now a first-class metric: judged verdicts carry a score: line (0..1 or 0..10, normalized) that becomes the AC's metric value, so baseline: delta forces per-round quality improvement. The template ships a quality-target AC (metric: score regex, expected: >=0.85) that gates GO; the loop's exit condition becomes: checks pass AND quality target met.

v0.4.1 — evidence binding is mandatory: untraceable judgments are inadmissible

Choose a tag to compare

@samirliu samirliu released this 10 Oct 07:08

Closes the last loophole in v0.4.0's evidence-fidelity work: verdicts WITHOUT artifact: + digest: sha256: lines are now voided to unverifiable. The back-compatible path (verdict keeps standing without evidence) was the same class of hole as the original miss — a judgment that cannot be traced to what it looked at is inadmissible.

Semantics now: judged verdict must cite the artifact it reviewed + its sha256; the gate re-hashes the cited artifact and voids the verdict if what ships no longer matches what was reviewed.

v0.4.0 — evidence fidelity: the gate can now tell 'checked the parts' from 'checked the thing that ships'

Choose a tag to compare

@samirliu samirliu released this 10 Oct 07:04

A real case proved the eval system had a structural blind spot: the gate returned GO (score 1.0) and a vision verdict said PASS while the actual deliverable was completely broken. Every AC verified data/derived artifacts; the review looked at renders, never the deliverable.

Generic upgrades (evidence fidelity):

  • deliverable: <path> in the contract declares the acceptance anchor; preflight warns deliverable-unverified (no check touches it) and judged-without-deliverable.
  • Judgment digest binding (R7 for judgments): verdicts citing artifact: + digest: sha256: are voided to unverifiable when the cited artifact no longer matches what ships.
  • no-runtime-ac warning: contracts that never execute the deliverable in any runtime.
  • Template + SKILL: verification pyramid — data → artifact → runtime → review of the deliverable itself.
  • Metric series identity = AC id + check hash (refinements keep history & no-regression; cross-contract id collisions impossible); bestScore resets only when the loop rolls over.

Suite green; every discipline locked by tests.

v0.3.1 — metric discipline (Goodhart leak closed)

Choose a tag to compare

@samirliu samirliu released this 10 Oct 06:11

Found by the multi-member Agent Teams case, on its first real multi-contract workspace:

The leak: an AC's metric dropped (new contract's AC-4 = 183 lines vs a stale baseline of 13938 from a previous contract). Judged failed once — then the failing observation itself slid the baseline to 183, and the next run 'passed' at the regressed level. The no-regression rule was defeated by its own baseline-update policy.

Two disciplines now enforced and tested:

  • Baselines advance only on a passed judgement (first observation still establishes; failed/unverifiable observations never lower the baseline).
  • Metric state (baselines/bestScore) is scoped to the contract stamp (R3): a contract change starts a fresh metric experiment — bare AC-id keys collide across contracts. Prior contracts grandfathered.

Also verified live in the same case: goal-only task-claim pass-through kept multi-member completions deadlock-free (partialCompletes=2 recorded).

v0.3.0 — loop-driver fallback CLI + eval self-hardening

Choose a tag to compare

@samirliu samirliu released this 10 Oct 05:39

Fixes found by running a real case (the Three.js 747 build) end-to-end:

Loop-driver fallback (bin/gate.mjs): the 747 loop was driven by hand-importing core.js because resumed sessions lack the goal_gate_* tools. Now any session can drive the loop from bash: bin/gate.mjs check|status|preflight --cwd <workspace> — same ledger, same output shape as goal_gate_check. Shared lib/ledger.js backs both entries.

Eval self-hardening: the gate once passed a visibly broken render — checks only measure what they are told. Two new defenses: preflight flags fragile check commands (bare node/python, relative interpreters, tautology suspects — the bare-node trap was a real near-miss), and neverFailed surfaces ACs that have never failed across ≥2 runs (suspected tautologies).

Tests: new ledger_test.mjs (preflint lint, never-failed detection, full CLI drive); suite green.

v0.2.5 — resolution chain fix: linked local installs now activate on DSH Desktop

Choose a tag to compare

@samirliu samirliu released this 10 Oct 04:32

Root cause (live-instrumented): the host's @deepseek-ai/* module routing does not serve link: local packages — the entry's bare import '@deepseek-ai/dsh-tools' failed natively, the module graph never instantiated, and apply() silently never ran (no tools, no commands, no interception). Every linked plugin on the host was equally dead; the composition was always clean.

Fix: a defineTool resolution chain — bare specifier (registry installs) → the installation's physical path via process.resourcesPath → identity shim fallback.

Live E2E (5/5 green): NO-GO deny with failing-AC names → BLOCKED false-complete ratchet → human reset + artifacts → digest-invalidation re-verify → GO; plus goal-only task-claim pass-through with milestone accounting that does not consume the round budget. Full evidence in docs/e2e-evidence.md.