Skip to content

feat(promotion): pool min_trades across same-parameter rules, with a diversity floor (#338) - #343

Merged
eaitbrahim merged 2 commits into
mainfrom
feat/338-pooled-min-trades
Aug 17, 2026
Merged

feat(promotion): pool min_trades across same-parameter rules, with a diversity floor (#338)#343
eaitbrahim merged 2 commits into
mainfrom
feat/338-pooled-min-trades

Conversation

@eaitbrahim

Copy link
Copy Markdown
Contributor

What & why

The promotion gate judged min_trades = 100 per rule per product. At the measured
daily rates (1.19–3.20 trades/asset-year, the go-live runbook's own table) a single
product needs 31–84 years to reach 100 trades — the floor was unreachable in a human
timespan, which is why the sandbox's five rules were live-seeded and why #337 adds an
hourly paper profile that makes samples collectable at all. This PR makes the gate able
to COUNT that evidence.

The change is to the gate's unit of evaluation, not its floors — 100 stays 100. This
is the agreement CONTRIBUTING requires for a gate change, recorded in the issue: the
operator approved it on 2026-08-17. The recorded discipline in promotion.py (the
win-rate axis was relaxed ALONE; axes move only with their own justification) is
respected: PromotionConfig is untouched.

Concretely, when promoting, sibling evidence is gathered — same kind, same params
(exact match on the stored JSON-plain form, minus product_id), different product_id,
status paper — plus the candidate itself, once. The sample-size axis passes if EITHER:

  • (a) the per-rule stat clears min_trades exactly as today (a rule with no
    siblings is judged byte-for-byte as before, and a rule whose own sample is full is
    still judged on its own stats — pooling is not a quality rescue for the sample-rich),
    OR
  • (b) pooled n ≥ 100 and a diversity floor: ≥ MIN_POOLED_PRODUCTS (5) distinct
    products each contributing ≥ MIN_TRADES_PER_PRODUCT_POOLED (10) trades.

The diversity floor is the honest discount on pooling: crypto assets correlate, so a
pool of correlated samples carries less information than its trade count claims —
pooled-but-correlated evidence overstates its statistical power. Requiring breadth
(products each with an independently meaningful sample) is how the pooled path pays for
the larger n instead of just collecting it.

Quality floors (expectancy / rr / win rate) are judged on the POOLED aggregates when
path (b) carries the decision; on the rule's own stats otherwise.

The PBO/overfitting gate (G4) stays per-rule, deliberately. It consumes the CSCV
result for the trial matrix the rule's parameters were selected from (--pbo-session) —
evidence about the parameter selection, which is per-parameter-set already — and this
PR does not change its scope. A pooled promotion without a run overfitting check still
refuses, exactly as before.

keel rules promote prints BOTH readings whenever a pool exists — per-rule n, pooled
n, and the per-product census — and failure reasons name their path:

rule 1 (pullback_continuation): sample readings -- per-rule n_trades=16, pooled n_trades=128 across 8 products
  pooled census (diversity floor 5 products x >= 10 trades): ADA-USD=16, BTC-USD=16, DOGE-USD=16, ETH-USD=16, LTC-USD=16, PAXG-USD=16, SOL-USD=16, XLM-USD=16 -- 8 products contribute, min contribution 16
rule 1 (pullback_continuation): overfitting check = pass
rule 1 (pullback_continuation): status -> paper

Pooling arithmetic (stated in pool_stats' docstring)

Pooled from the per-result aggregates BacktestResult actually carries: pooled n = Σn;
per-result wins recovered as round(n × win_rate) (the float round-trip error of
wins/n is ~1e-16, far below 0.5); pooled win rate = Σwins/Σn; pooled avg_win/avg_loss
= win-/loss-weighted means; pooled expectancy/profit_factor/avg_mfe/avg_mae = trade-
weighted means / gross sums with summarize's conventions. max_drawdown and
max_losing_streak are NOT pooled — they are path-dependent and unrecoverable from
aggregates — and are set to 0; the gate reads neither.

One adjacent fix

transition()'s kind-level row lookup ("newest row of this kind") predates multi-row
kinds; with pools made of same-kind sibling rows it would have advanced a sibling the
operator never typed. keel rules promote now pins the target row (rule_id=); the
kind-level lookup remains for library callers that don't name a row (pinned by test).

Closes #338

Tests-first evidence

  • Tests written first, seen failing for the right reason

Red run (tests written before any implementation):

$ uv run pytest tests/strategy/test_promotion.py -q
ImportError: cannot import name 'MIN_POOLED_PRODUCTS' from 'keel.strategy.promotion'
1 error during collection

$ uv run pytest tests/test_cli.py -q -k "pooled or siblings or readings"
test_rules_promote_reports_both_readings_and_promotes_via_the_pooled_path FAILED
  AssertionError: assert 'per-rule n_trades=16' in <output>   # the command printed no readings
test_rules_promote_names_the_diversity_failure_when_the_pool_is_too_narrow FAILED
  AssertionError: assert 'pooled n_trades=120 across 4 products' in <output>
1 failed, 1 passed, 75 deselected   # then: 2 failed after the module tests went red on import

New coverage: single-product pass/fail unchanged (including "no pooled lines at all");
pooled pass (8×16=128, candidate's own quality fails, pooled quality carries it); pooled
fail on n (8×10=80, reason names path+census); pooled fail on diversity (4×30=120 but
only 4 products); pooled quality judged on pooled aggregates; a full own-sample rule not
rescued by the pool; pooled reading reported even when per-rule passes; params-mismatch
and non-paper rows are not siblings; candidate's own product never double-counted;
duplicate sibling rows pool once; transition promotes via the pooled path and still
refuses without a run overfitting check; CLI prints both readings + census.

Gates (all must pass)

  • uv run ruff check clean
  • uv run mypy clean
  • uv run pytest -q green
2908 passed, 1 skipped in 38.87s
All checks passed!            # ruff
Success: no issues found in 237 source files   # mypy

Scope check

  • This PR touches a rail or a default classification — checked means it DOES;
    leave checked only if true, and if so: cite the source and open the discussion
    BEFORE review (CONTRIBUTING.md, "Governance: rulings vs. machinery").
  • New dependency added (needs discussion first)

…, scratch caveat stated, pooled-pass named in words, str-vs-int mismatch case
@eaitbrahim
eaitbrahim merged commit 6cf1be4 into main Aug 17, 2026
5 checks passed
@eaitbrahim
eaitbrahim deleted the feat/338-pooled-min-trades branch August 17, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pool min_trades across same-parameter rules, with a cross-sectional diversity floor

1 participant