Skip to content

feat(hmm): add historic backtest runner script for HMM + StrategyOrchestrator #3

@Pepeye

Description

@Pepeye

Goal

Stand up a minimal CLI script that lets a user run the HMM + StrategyOrchestrator stack on historic OHLCV data and see how it would have performed — without wiring the full backtesting.py adapter.

Why

shared/core/hmm/ is fully decoupled from backtest/. The only way to see the HMM interact with real price series today is to write bespoke glue. A reusable walk-forward script closes that gap and serves as a smoke test for the whole HMM pipeline.

Scope

scripts/hmm_backtest.py — CLI with:

  • --ticker (default SPY), --start, --end, --interval (default 1d), --warmup-pct (default 0.5)
  • Fetches OHLCV via YFinanceFetcher
  • Runs prepare_features_for_hmm on the full series, splits into train / out-of-sample at the warmup boundary
  • Trains HMMEngine on the warmup features
  • Walks forward through OOS bars calling predict_filtered(feature_vector, timestamp=bar_time) — never model.predict() (no look-ahead)
  • For each OOS bar, asks StrategyOrchestrator.should_enter_position for target position_size + leverage
  • Computes bar-to-bar equity: equity *= 1 + exposure * bar_return
  • Prints: Final equity, Total return %, Annualised Sharpe, Max drawdown, Bar count, Regime distribution

tests/hmm/test_backtest_runner.py — pytest using the synthetic sample_ohlcv_data fixture. Validates:

  • Equity curve is finite and positive
  • No look-ahead: runner never calls model.predict
  • Output summary contains all required fields

Live smoke runuv run python scripts/hmm_backtest.py --ticker SPY --start 2020-01-01 --end 2024-12-31 exits 0 and prints the summary block.

Non-goals

  • Slippage / transaction cost modelling
  • Full HMMBacktestStrategy adapter on top of backtesting.py
  • Stop-loss / take-profit execution simulation
  • Multiple tickers / portfolio construction

Acceptance criteria

  • scripts/hmm_backtest.py exists with argparse CLI
  • Core logic factored into an importable function so pytest can exercise it on synthetic data without touching the network
  • Unit test in tests/hmm/test_backtest_runner.py passes with the synthetic fixture
  • Script uses predict_filtered, not model.predict — look-ahead-bias-free
  • uv run ruff check scripts tests/hmm exits 0
  • uv run ruff format --check scripts tests/hmm exits 0
  • uv run ty check scripts tests/hmm reports no in-scope errors
  • uv run python -m pytest tests/hmm -q shows all tests passing
  • Live smoke run on SPY 2020–2024 daily bars exits 0 and prints Final equity / Total return / Sharpe / Max drawdown / Regime distribution

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature requesthmmHidden Markov Model related

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions