Skip to content

Agent ETA v0.6.0 — Frozen cohorts and future validation windows

Pre-release
Pre-release

Choose a tag to compare

@maxi-max-dev maxi-max-dev released this 11 Oct 00:14

Agent ETA v0.6.0 adds reproducible separation of old tuning runs from a predeclared future run window. It keeps the displayed ETA and default model unchanged.

  • cohort-plan reads an existing local database, freezes completed tuning runs and their saved receipt IDs, and emits a manifest with future UTC start/end bounds.
  • compare --plan FILE --cohort tuning|holdout assigns complete runs by start time, preserves pending and cross-window long runs, rejects edited/deleted frozen rows, and ignores predictions added later to tuning runs.
  • Reports include profile/class and fixed active-duration strata, sparse-history abstentions, missing/invalid records, and all cohort exclusions. Unplanned compare and evaluate remain compatible with v0.5.0.
  • The time-split protocol was frozen in commit 5f2c702 before implementation or inspection of a target cohort's results. Usage is documented in both READMEs and docs/TIME-SPLIT.md.

Validation: 263 local tests, 12 cohort-specific tests, clean tarball installation and CLI checks, byte-identical unplanned reports versus v0.5.0 on the same synthetic fixture, unchanged database/WAL checks, and Linux/Windows × Node 22/24 public CI.

This is an experimental release of evaluation tooling. Synthetic fixtures and controlled clocks verify behavior, not real forecast accuracy. A manifest/hash does not prove that a holdout is real or untouched. Promotion gates and an independently documented future real cohort remain pending; promotionEligible stays false. Keep local manifests, reports and databases private. No data migration, upload, deletion, new background service or model promotion is introduced.

中文:本版把旧样本与未来新运行按事先约定的时间切开,补齐任务长度和类型分层,防止事后补预测或只挑已经结束的好看样本。测试通过不等于预测更准;默认模型保持原样,下一步是先冻结晋升门槛,再收集独立未来实测。