Engine: persist comprehensive gauntlet artifacts (Monte Carlo quantiles, DSR inputs and score, full CSCV result) when the gauntlet runs

[eaitbrahim]

The defect, in one line

keel computes evidence and keeps only the prose. The gauntlet runs, prints a report, appends a row carrying a handful of scalars — and the distributions, the matrix and the inputs that produced them are gone the moment the process exits.

This is the same defect #708's Evidence Matrix is blocked on (per-cell sweep results written to an untracked docs/experiments/_out/), seen from the gauntlet side. Both surface as "the UI cannot show this", but neither is a UI problem.

What is recorded today

Six rows in docs/experiments/trials-ledger.jsonl carry gauntlet fields at all, out of 93:

trial	pbo	pbo_available	gate_passed	seed	bars
476-optuna-turtle_breakout	0.7	1	0	476	17520
476-optuna-rsi_meanrev	0.571…	1	0	476	17520
476-optuna-pullback_continuation	0.842…	1	0	476	17520
cusum-event-first-measurement-2026-09-01	—	0	0	—	—
triple-barrier-first-measurement-2026-09	—	0	0	—	—
per-product-slippage-restatement-2026-09	—	0	0	—	—

That is genuinely useful and #708's view 3 is being built to display it. What it cannot support is anything distributional.

What is NOT recorded, and what each absence costs

Monte Carlo distributions. research/montecarlo.py produces a resampled distribution and a percentile; the ledger keeps neither the quantiles nor the array. #708 asks for "Monte Carlo histogram as pre-computed SVG polylines (the server-quantised pattern the equity curve already uses — zero client-side math)". There is nothing stored to quantise. Recomputing means re-running a backtest inside a web request.

DSR / E[max SR] / MinBTL. keel trials deflate takes --sharpe as a required operator input — the ledger stores no observed annualised Sharpe per trial. So DSR is not merely expensive to recompute; it is impossible without synthesising an input, which is the one thing this codebase refuses. Record observed_annual_sharpe, skewness, kurtosis, trades_per_year and trial_sharpe_variance (or an explicit null where it could not be supplied) at the moment the operator runs the command with them, and DSR becomes a read.

The CSCV matrix and its result. research/cscv.py returns pbo, degradation_slope, prob_loss, dominance_1st, dominance_2nd, n_columns, n_blocks, n_combinations, rows_used, rows_dropped. One scalar of those ten reaches the ledger.

Why this cannot be papered over at read time

Measured on the real ledger, per session:

pbo-grid-entry-lookback-2026-07-20   12 columns   PBO 0.881   11.9 s
pbo-grid-repaired-2026-07-20         12 columns   PBO 0.893   12.9 s
exit-lookback-2026-07-20              5 columns   PBO 0.925   14.3 s
optuna-parameter-study-2026-08-22      0 columns   (too thin for CSCV)

~39 s of CPU for three sessions, on a page whose POLL_MS is 15 s.

And over the ledger as a whole it does not run at all:

ValueError: columns are not synchronous: found lengths [1819, 1828];
§78.6 requires a true matrix with the same rows for every column

build_matrix requires synchronous columns, so PBO is only ever defined within a session whose trials share a bar count. A page cannot pick that scope without inventing an operator's decision.

Proposed scope

When a gauntlet component runs — trials pbo, trials deflate, trials monte-carlo, trials walk-forward, and the promotion path that invokes them — serialise its full result into the ledger row rather than printing it and discarding it:

• CSCV: every field of PBOResult, plus session and the column count the matrix was built from.
• Monte Carlo: the quantiles (a fixed ladder — p1, p5, p25, p50, p75, p95, p99), the observed value's percentile, the resampling method, the seed, and the draw count. Quantiles rather than the raw array: a stored ladder is bounded, is what a chart needs, and does not turn an append-only text file into a data dump.
• DSR: the inputs (observed_annual_sharpe, skewness, kurtosis, trades_per_year, trial_sharpe_variance) alongside the outputs (dsr, expected_max_sharpe, min_trades), so the number can be recomputed and checked rather than trusted.

No backfill. The six existing rows keep exactly the fields they have, and the UI keeps reading "not recorded" for the rest. A gauntlet result reconstructed after the fact is a number nobody ran, which is worse than an honest gap — the same posture #721 settled on for the audit chain.

⛔ The Strathern rail applies to every field above. These are diagnostics. cscv.py, deflate.py and walkforward.py each carry the rail comment already: a score may report, and may gate, but may never be a sweep's ranking key. Storing them makes them easier to rank by, so the rail needs restating wherever they are read — PBOResult deliberately carries no configuration field, and nothing stored here should either.

Sequencing

Batches naturally with the Schema v20 engine sprint (#715, #718, #719, #721) — but note this one touches keel/research/ledger.py's row shape rather than the SQLite schema, so it needs no migration, only a widened summary convention and writers that populate it.

What this unblocks

#708 view 3's distributional half (the Monte Carlo polylines and the DSR scorecard), and it is a prerequisite for any honest answer to "how overfit is this?" that does not cost 12 seconds and a re-run.