Skip to content

Releases: HuangShengZeBlueSky/dsh-harness-evolution

v0.4.0 · Methods, curves and reproducible DSH loops

Choose a tag to compare

@HuangShengZeBlueSky HuangShengZeBlueSky released this 30 Oct 09:07

Methods, curves, and reproduction / 方法、曲线与复现

This release links 25 method definitions to 115 recorded campaign/stage curves, five benchmark artifacts, explicit configurations and native test statistics. It adds a bilingual interactive research explorer, 36 three-benchmark L1–L9 profiles, 44 method-final harnesses with diagrams, and the retrospective OSBench L4 v03 artifact (170/432, 39.35%). Original final selections remain unchanged.

The audit makes protocol differences explicit: L4/L5 and L6/L7–L9 change task/validation pools as well as strategy. New OS mechanism feedback improves average success but not all-three repeat success. Experience delivery, intended repair and global benefit remain separate measurements. Historical Skill/Workflow and Astra/Luna recipes retain their reproduction limits.

公开内容包括:方法的一句话摘要与逐步配方、训练/验证/测试分开的曲线、每次测试与标准差、三次任意成功/全部成功、配对修复与退步、实际运行计数和可用耗时。未知成本保留为空,不把测试重复当作独立进化重复。Telecom公开内置Life-Harness的GLM复测单独记录为57.50%,不冒称论文90分产物的同协议复现。

Validation: the shared L7 scripted fixture exercises actual DSH hooks through plan, evaluation, adoption and final test. Published plugin imports and data builds pass. The cleaned package has not rerun all historical benchmarks; ALFWorld/Telecom historical search and feedback extensions have explicit integration limits. No new benchmark experiments were launched for this publication.

Interactive explorer · 中文方法 · Reproduction

v0.3.0 · Standout checkpoints, method diagrams and measured curves

Choose a tag to compare

@HuangShengZeBlueSky HuangShengZeBlueSky released this 27 Oct 02:57

Three newly released promoted checkpoints, with complete retrospective measurements:

Benchmark Artifact Success Coverage
WebShop Original method-1 continuation v12 47.00% strict; 71.21 mean reward / 100 600 episodes
DBBench L4-H0 v32 70.33% 900 episodes
OSBench L5-H0 v08 38.66% 432 episodes

All three were genuinely adopted during training. They are retrospectively selected releases, not replacements for the original training-final decisions. The leaderboard now separates best measured promoted checkpoints from training-selected final artifacts; WebShop's published training-final best remains 44.50%.

  • Eight runnable artifact bundles across five benchmarks.
  • Separate method-flow and runtime-architecture Mermaid diagrams, with bilingual method diagrams and artifact documentation.
  • Evolution curves enriched only with completed backfill test points. Unmeasured checkpoints remain blank.
  • Original final artifacts and historical/current paper references remain available.

These are completed original-run measurements, not a fresh evaluation of the portable packages. Actor/trial/protocol differences from paper references remain documented.

中文首页 · English · 完整榜单

v0.2.0 · Bilingual leaderboard and aligned publication

Choose a tag to compare

@HuangShengZeBlueSky HuangShengZeBlueSky released this 26 Oct 04:16

Bilingual publication and leaderboard refinement. Benchmark scores are unchanged; this is a packaging and reporting update, not a new benchmark run.

  • Five published harnesses with Chinese/English artifact pages, Mermaid diagrams and measured evolution curves.
  • Multi-method reference tables and historical/current Life-Harness Telecom reports retained together.
  • Primary champions selected within registered evaluation groups; incompatible protocols cannot silently enter the same ranking.
  • Published scores checked against the actual selected checkpoint and complete test coverage.
  • WebShop's development curve now shows the strict-success objective used by L5, with adoption markers and the released checkpoint labeled.
  • Publication preflight (--check) and ordinary YAML date support; Chinese running and publishing guides added.

中文首页 · English · Leaderboard

v0.1.0 · Five evolved harnesses

Choose a tag to compare

@HuangShengZeBlueSky HuangShengZeBlueSky released this 26 Oct 03:49

First public release: executable evolved DSH harnesses for five benchmarks.

Benchmark Checkpoint Success
ALFWorld v18 95.41%
DBBench v35 70.56%
WebShop v28 44.50% strict success; 70.68 mean reward
OSBench v17 38.89%
τ²-Telecom v06 84.17% Pass@1; 100% Pass@3; 62.50% Pass³

Each artifact includes native plugins, composition, measured protocol, Mermaid architecture and recorded evolution curve. The repository includes a publication command and an automatic leaderboard-rendering workflow. Prior releases are retained.

These are original completed-run measurements, not a new evaluation of the portable package. Paper reference values and local reference runs are separate. Telecom uses a different user simulator plus account-facts and auxiliary-review integration; its score is not a direct Life-Harness comparison.

See leaderboard and curves.