Skip to content

keel research: thirteen evidence modules with no front doorΒ #601

Description

@eaitbrahim

Jesse's two marquee statistical features are shallower than what keel already ships β€” and keel's have no front door.

Jesse's homepage gives two feature cards to statistical validation:

πŸ”¬ Rule Significance Testing β€” "Bootstrap resampling helps determine whether a rule's historical edge is genuine or could have appeared by chance."
🎲 Monte Carlo Analysis β€” "trade-order shuffling and candles-based simulations to distinguish skill from luck."

keel/research/ already contains both, plus things Jesse does not have β€” thirteen modules on main:

module what it does Jesse
significance.py one-proportion test against break-even priced at the fee actually paid (#475) marketed equivalent
montecarlo.py trade reshuffle and candle bootstrap (#441) marketed equivalent
cscv.py Probability of Backtest Overfitting via CSCV β€”
deflate.py Deflated Sharpe, E[max SR_n], Minimum Backtest Length β€”
walkforward.py rolling-origin validation of a given parameter set (#445) partial (train/test split)
independence.py "two rules that fire together are one rule counted twice" β€”
throughput.py allowance-throughput planning (#478) β€” how long the evidence honestly takes inside fee-free allowances β€”
cts_factors.py do the 11 CTS confluence factors carry independent evidence, or one momentum read counted thrice (#208) β€”
ledger.py the trials budget and its hash-chained provenance β€”
pooled_review.py the underpowered-n honesty discipline, in report form β€”
bias.py, matrix.py, tuning.py bias census, the ruleΓ—venue matrix, declared parameter spaces β€”

keel's versions are also more honest. significance.py's docstring: "a significance tool here must be able to say 'not distinguishable from zero' and mean it. A tool that cannot say no is a flattery tool." And montecarlo.py names the invariant Jesse's marketing does not β€” a permutation of a multiset sums to the same number, so every reshuffled path ends at the same final equity and that percentile is exactly 1/2 by construction.

Nothing in the category has the Strathern rail (cscv.py, deflate.py, walkforward.py): a score may report, and may gate, but may never be a sweep's ranking key β€” pinned by a test that fails if a configuration-bearing field is added.

The gap

Thirteen modules with no single entry point is operationally the same as not having them. There is no keel research command on main, nothing in the README, nothing on the site.

In-flight overlap (2026-08-28): a keel/commands/research_console.py already exists on the unmerged docs-web-ui-spec work branch. Reconcile with that work before implementing β€” the front door gets built once, and the acceptance criteria below apply to whichever console lands.

The ask

  • keel research as one front door over the existing modules β€” subcommands, no new statistics
  • Every subcommand must be able to return the refusal, and print it as a result rather than an error
  • A docs page that states what each answers, and β€” importantly β€” what it cannot answer
  • The Strathern rail explained where a reader will meet it, since it is the part nobody else has
  • No new metrics in this issue. Surfacing only. If a gap appears, file it separately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

researchMeasurement, validation or KB work (Research & validation)

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions