Repository navigation
v0.3.2 -- Backtest harness: charter empirical gate
v0.3.2 -- Backtest harness: the charter's empirical gate
The release the charter has been waiting for.
Charter: "if your three-analyst committee on BTC can't beat buy-and-hold
risk-adjusted on paper, more analysts won't fix it."
Until now that question required 4-8 weeks of wall-clock paper trading
to answer. v0.3.2 makes it computable in seconds via historical replay
through the production advisor pipeline.
hermes quant backtest --symbol BTC/USDT --asset-class crypto
--timeframe 1h --bars-file ~/.hermes/quant/cache/btc-1h.csv
Output: report.md + result.json + equity_curve.csv + decisions.jsonl
in ~/.hermes/quant/backtests//.
The line that matters: "Excess return vs buy-and-hold: +X.XX%"
Highlights
- ADR-0020: REPLAY (not simulation). Uses production advisor pipeline
verbatim. Charter Reproducibility honored: same bars + same config_hash
-> same equity curve, byte-identical. - backtest.replay() walks bars chronologically through advisor.recommend()
with lookahead-safe as_of cutoff at every bar. - PaperPortfolio: minimal mark-to-market book with slippage, commission,
lot-matching realized P&L, position flips. - Buy-and-hold baseline + Sharpe + Deflated Sharpe (PSR for n_trials=1)
- max drawdown computed inside replay().
- BacktestResult.excess_return_vs_buy_hold_pct is THE charter-gating
headline. Negative over multi-month backtest = "fix analysts before
RL aggregator work" per charter.
Tests: 530 passed, 1 skipped (was 506 -> +24, zero regressions).
What this unblocks
v0.4+ RL aggregator work was charter-gated on "paper-trade for 4-8
weeks first." With replay, operators can now answer:
- Does the three-analyst committee beat buy-and-hold on BTC/USDT 1h
over 2024? Run a backtest. (Seconds.) - What's the Deflated Sharpe accounting for backtest-overfitting? In
the report. - What's the per-bar equity curve? equity_curve.csv ready to plot.
The charter's empirical gate is now computable. v0.4's RL work can
proceed iff the v0.3.x committee actually beats buy-and-hold on
real data -- which is exactly what the charter requires.