Metadata
| Field |
Value |
| CodeClone version |
2.1.0a1 (reproduced on release/2.1.0a2, commit 932d4b73) |
| Affected area |
CLI / baseline / cache |
What happened?
Baseline trust enforcement under --ci is asymmetric between the clone lane and the metrics lane. An untrusted/missing clone baseline fails closed (exit 2), but an untrusted/missing metrics baseline under bare --ci silently disables all metrics-delta gates and the run exits 0.
Evidence:
--ci forces fail_on_new = True unconditionally (codeclone/surfaces/cli/startup.py:85-91) → any untrusted clone baseline becomes ExitCode.CONTRACT_ERROR (2).
--ci forces fail_on_new_metrics = True only if the metrics baseline actually loaded: if args.ci and state.loaded: args.fail_on_new_metrics = True (codeclone/surfaces/cli/baseline_state.py:247-248). Missing/untrusted metrics section → the flag stays off → no contract error is raised for that lane.
- With
metrics_diff = None, summarize_metrics_diff(None) returns None and every diff_* gate field defaults to 0 (codeclone/report/gates/evaluator.py:75-77,166) — all delta gates (health regression, typing/docstring regression, new cycles, new dead code, API breaks) evaluate as "no regression".
- Bonus dead parameter:
evaluate_gates() accepts baseline_status and discards it (evaluator.py:547: _ = baseline_status), so gate evaluation cannot see baseline trust at all.
Net effect: in CI, tampering with or deleting the embedded metrics section (or a schema mismatch after upgrade) does not fail the pipeline — it just switches the metrics gates off. Confused evidence becomes a confident green.
What did you expect instead?
Symmetric fail-closed gating: under --ci/gating, any required lane that is untrusted/missing exits 2 before gate evaluation — same policy for clones and metrics. An untrusted lane must never silently disable its gates; in normal (non-gating) mode it should be reported as unavailable, not as "no regression".
Minimal reproduction
# 1. Create a trusted baseline:
# codeclone . --update-baseline
# 2. Corrupt only the metrics section (or its metrics_payload_sha256) in
# codeclone.baseline.json, leaving clones intact.
# 3. Run: codeclone . --ci
# Observed: warning about untrusted metrics baseline, exit code 0,
# all metrics gates skipped.
# Expected: exit code 2 (contract error), same as a corrupted clone section.
Additional context
- Explicit per-gate flags (
--fail-on-new-metrics, --fail-on-typing-regression, ...) do pre-empt the hole via _metrics_baseline_gate_requested (baseline_state.py:293-350,398-404) — the asymmetry is specifically the bare --ci path.
- Fix contract: Phase 39 spec
specs/phase-39-baseline-container-lanes.md §10.11 + invariant I-16 (gating symmetry; per-lane trust vector; evaluate_gates must consume trust).
- Related UI symptom: HTML overview uses metrics-diff presence as the "baselined" proxy for clone novelty display (
report/html/sections/_overview.py:900-930).
Metadata
release/2.1.0a2, commit932d4b73)What happened?
Baseline trust enforcement under
--ciis asymmetric between the clone lane and the metrics lane. An untrusted/missing clone baseline fails closed (exit 2), but an untrusted/missing metrics baseline under bare--cisilently disables all metrics-delta gates and the run exits 0.Evidence:
--ciforcesfail_on_new = Trueunconditionally (codeclone/surfaces/cli/startup.py:85-91) → any untrusted clone baseline becomesExitCode.CONTRACT_ERROR(2).--ciforcesfail_on_new_metrics = Trueonly if the metrics baseline actually loaded:if args.ci and state.loaded: args.fail_on_new_metrics = True(codeclone/surfaces/cli/baseline_state.py:247-248). Missing/untrusted metrics section → the flag stays off → no contract error is raised for that lane.metrics_diff = None,summarize_metrics_diff(None)returnsNoneand everydiff_*gate field defaults to0(codeclone/report/gates/evaluator.py:75-77,166) — all delta gates (health regression, typing/docstring regression, new cycles, new dead code, API breaks) evaluate as "no regression".evaluate_gates()acceptsbaseline_statusand discards it (evaluator.py:547:_ = baseline_status), so gate evaluation cannot see baseline trust at all.Net effect: in CI, tampering with or deleting the embedded
metricssection (or a schema mismatch after upgrade) does not fail the pipeline — it just switches the metrics gates off. Confused evidence becomes a confident green.What did you expect instead?
Symmetric fail-closed gating: under
--ci/gating, any required lane that is untrusted/missing exits 2 before gate evaluation — same policy for clones and metrics. An untrusted lane must never silently disable its gates; in normal (non-gating) mode it should be reported asunavailable, not as "no regression".Minimal reproduction
Additional context
--fail-on-new-metrics,--fail-on-typing-regression, ...) do pre-empt the hole via_metrics_baseline_gate_requested(baseline_state.py:293-350,398-404) — the asymmetry is specifically the bare--cipath.specs/phase-39-baseline-container-lanes.md§10.11 + invariant I-16 (gating symmetry; per-lane trust vector;evaluate_gatesmust consume trust).report/html/sections/_overview.py:900-930).