Skip to content

feat: baseline sanity gate, standing-failure delta, health_command - #84

Merged
geuben merged 23 commits into
mainfrom
feat/67-baseline-sanity
Aug 28, 2026
Merged

feat: baseline sanity gate, standing-failure delta, health_command#84
geuben merged 23 commits into
mainfrom
feat/67-baseline-sanity

Conversation

@geuben

@geuben geuben commented Aug 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • Infra-down gate (Part 1, cycles 1–6): run start now refuses a baseline where more than 50% of a suite with ≥10 collected tests is failing — the pattern observed when backing services are down. Returns reason: baseline_implausible. Pass --accept-baseline to override; the override records a baseline_accepted integrity event. Per-project baseline_max_failure_ratio in tdd.toml tunes the threshold.
  • Standing-failure delta (Part 2, cycles 7–9): After capturing a non-empty baseline, run start emits a baseline_standing_delta event partitioning standing failures into new / inherited / resolved against the previous run's baseline for that worktree, making a permanently-red set visible as a trend instead of ambient noise.
  • Environment-dependent suites (Part 3, cycles 10–12): A project can declare health_command in tdd.toml; run start runs it before probing and refuses with reason: services_unreachable on non-zero exit — the precise diagnosis rather than a ~1000-failure baseline being silently recorded.

Commits

61a163d docs: friction log for issue-67-baseline-sanity
c5208e1 docs: document baseline sanity gate, standing-failure delta, health_command
d46fcd5 test: pin that a passing health_command does not block the run
adff5be feat: probe health_command before baseline capture, refuse if down
ceeea32 test: a failing health_command refuses with services_unreachable
c3216db feat: Project.health_command parsed and validated from tdd.toml
55fa896 test: a project's health_command parses onto Project
27d1583 test: pin the no-prior-baseline delta path
75f883b feat: emit baseline_standing_delta partitioning new vs inherited
8a3ff0c test: a non-empty baseline logs new-vs-inherited standing failures
719069d feat: Ledger.previous_baseline joins baseline to run by worktree
8fe7f41 test: previous_baseline reads the last earlier run's failing set
3514380 test: pin that a healthy baseline is untouched by the gate
e7a8cf3 feat: emit a baseline_accepted integrity event on override
91e63d1 test: an accepted implausible baseline logs baseline_accepted
bfd4827 feat: honour per-project baseline_max_failure_ratio from tdd.toml
b7cd4b8 test: a project's baseline_max_failure_ratio suppresses the refusal
4cdf24d test: pin the small-suite exemption from the baseline gate
23ddac8 feat: --accept-baseline bypasses the implausibility gate
a1b1fc4 test: --accept-baseline lets an implausible baseline through
c003f69 feat: refuse a baseline whose failing ratio exceeds the threshold
ac807aa test: run start refuses an implausible mostly-red baseline

Quality gates

  • uv run ruff check src tests — passed
  • uv run pytest — 418 passed

Test coverage note

All decision-point operators in new code are covered by at least one test (pseudo-mutation analysis passed). Sensitivity checks for all four pin cycles confirmed each boundary is detectable.

Closes #67

geuben added 22 commits August 28, 2026 11:59
TDD-Run: 11
TDD-Cycle: 1
TDD-Phase: red
TDD-Run: 11
TDD-Cycle: 2
TDD-Phase: red
TDD-Run: 11
TDD-Cycle: 2
TDD-Phase: green
TDD-Run: 11
TDD-Cycle: 3
TDD-Phase: pin
TDD-Run: 11
TDD-Cycle: 5
TDD-Phase: red
TDD-Run: 11
TDD-Cycle: 5
TDD-Phase: green
TDD-Run: 11
TDD-Cycle: 6
TDD-Phase: pin
TDD-Run: 11
TDD-Cycle: 7
TDD-Phase: green
TDD-Run: 11
TDD-Cycle: 8
TDD-Phase: green
TDD-Run: 11
TDD-Cycle: 9
TDD-Phase: pin
TDD-Run: 11
TDD-Cycle: 10
TDD-Phase: red
TDD-Run: 11
TDD-Cycle: 10
TDD-Phase: green
TDD-Run: 11
TDD-Cycle: 12
TDD-Phase: pin
@geuben
geuben merged commit 042a7d8 into main Aug 28, 2026
14 checks passed
@geuben geuben mentioned this pull request Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

run start: baseline capture records environment-broken baselines as legitimate

1 participant