Repository navigation
Releases: HuangShengZeBlueSky/dsh-harness-evolution
Release list
v0.4.0 · Methods, curves and reproducible DSH loops
Methods, curves, and reproduction / 方法、曲线与复现
This release links 25 method definitions to 115 recorded campaign/stage curves, five benchmark artifacts, explicit configurations and native test statistics. It adds a bilingual interactive research explorer, 36 three-benchmark L1–L9 profiles, 44 method-final harnesses with diagrams, and the retrospective OSBench L4 v03 artifact (170/432, 39.35%). Original final selections remain unchanged.
The audit makes protocol differences explicit: L4/L5 and L6/L7–L9 change task/validation pools as well as strategy. New OS mechanism feedback improves average success but not all-three repeat success. Experience delivery, intended repair and global benefit remain separate measurements. Historical Skill/Workflow and Astra/Luna recipes retain their reproduction limits.
公开内容包括:方法的一句话摘要与逐步配方、训练/验证/测试分开的曲线、每次测试与标准差、三次任意成功/全部成功、配对修复与退步、实际运行计数和可用耗时。未知成本保留为空,不把测试重复当作独立进化重复。Telecom公开内置Life-Harness的GLM复测单独记录为57.50%,不冒称论文90分产物的同协议复现。
Validation: the shared L7 scripted fixture exercises actual DSH hooks through plan, evaluation, adoption and final test. Published plugin imports and data builds pass. The cleaned package has not rerun all historical benchmarks; ALFWorld/Telecom historical search and feedback extensions have explicit integration limits. No new benchmark experiments were launched for this publication.
v0.3.0 · Standout checkpoints, method diagrams and measured curves
Three newly released promoted checkpoints, with complete retrospective measurements:
| Benchmark | Artifact | Success | Coverage |
|---|---|---|---|
| WebShop | Original method-1 continuation v12 | 47.00% strict; 71.21 mean reward / 100 | 600 episodes |
| DBBench | L4-H0 v32 | 70.33% | 900 episodes |
| OSBench | L5-H0 v08 | 38.66% | 432 episodes |
All three were genuinely adopted during training. They are retrospectively selected releases, not replacements for the original training-final decisions. The leaderboard now separates best measured promoted checkpoints from training-selected final artifacts; WebShop's published training-final best remains 44.50%.
- Eight runnable artifact bundles across five benchmarks.
- Separate method-flow and runtime-architecture Mermaid diagrams, with bilingual method diagrams and artifact documentation.
- Evolution curves enriched only with completed backfill test points. Unmeasured checkpoints remain blank.
- Original final artifacts and historical/current paper references remain available.
These are completed original-run measurements, not a fresh evaluation of the portable packages. Actor/trial/protocol differences from paper references remain documented.
v0.2.0 · Bilingual leaderboard and aligned publication
Bilingual publication and leaderboard refinement. Benchmark scores are unchanged; this is a packaging and reporting update, not a new benchmark run.
- Five published harnesses with Chinese/English artifact pages, Mermaid diagrams and measured evolution curves.
- Multi-method reference tables and historical/current Life-Harness Telecom reports retained together.
- Primary champions selected within registered evaluation groups; incompatible protocols cannot silently enter the same ranking.
- Published scores checked against the actual selected checkpoint and complete test coverage.
- WebShop's development curve now shows the strict-success objective used by L5, with adoption markers and the released checkpoint labeled.
- Publication preflight (
--check) and ordinary YAML date support; Chinese running and publishing guides added.
中文首页 · English · Leaderboard
v0.1.0 · Five evolved harnesses
First public release: executable evolved DSH harnesses for five benchmarks.
| Benchmark | Checkpoint | Success |
|---|---|---|
| ALFWorld | v18 | 95.41% |
| DBBench | v35 | 70.56% |
| WebShop | v28 | 44.50% strict success; 70.68 mean reward |
| OSBench | v17 | 38.89% |
| τ²-Telecom | v06 | 84.17% Pass@1; 100% Pass@3; 62.50% Pass³ |
Each artifact includes native plugins, composition, measured protocol, Mermaid architecture and recorded evolution curve. The repository includes a publication command and an automatic leaderboard-rendering workflow. Prior releases are retained.
These are original completed-run measurements, not a new evaluation of the portable package. Paper reference values and local reference runs are separate. Telecom uses a different user simulator plus account-facts and auxiliary-review integration; its score is not a direct Life-Harness comparison.
See leaderboard and curves.