Refutation Harness

A strategy-falsification harness for intraday trading signals. Its headline output is a negative result: after ~2,700 configurations over 6 months of 5-minute data across 12 tickers, it concluded that no tested signal had a tradeable edge — and the project was closed on that verdict.

Most backtesting repos exist to show a strategy working. This one exists to show how a strategy that looks like it works gets taken apart. The full evidence is in VERDICT.md — the centrepiece of this repository — including the control experiment in which a "confirmed" edge (walk-forward-validated and passed by an adversarial LLM verifier) beat a random-entry baseline by only 0.04 profit factor — noise.

Entry (identical trailing-stop exit, held-out test)	PF
mean-reversion (the "confirmed edge")	1.236
always-long	1.167
always-short	1.183
random direction	1.199

The apparent edge was the exit harvesting generic intraday structure; the entry signal contributed noise. No entry signal → nothing to trade.

Methodology

The harness stacks every safeguard that made that conclusion trustworthy:

• No-look-ahead invariant — rolling breakout channels and volume baselines are shift(1)-ed so a bar is never part of its own reference; trailing stops only use extremes from prior bars. Guarded by a regression test (tests/test_strategy.py).
• Anchored walk-forward out-of-sample validation (walkforward.py) — parameters are re-fit each window on an expanding train set and applied to the next, untouched block; every test block is stitched into one OOS track record. Splits are on whole trading days.
• Curve-fit-resistant optimisation (optimize.py) — two-stage grid search scored by the worst of the train/test scores, so a combo that only shines on the data it was tuned on loses.
• Explicit cost ladder (config.CostParams, option_data.py) — bid/ask spread, slippage, per-contract fees, and execution lag, all defaulting to 0.0 so the frictionless baseline reproduces exactly, then layered on with real quoted spreads for the costed run.
• Signal-free controls (controls.py) — always-long, always-short, and seeded-random entries that run through the same exit/cost/sizing engine via run_backtest(df, sym, signal_fn=...). If a strategy cannot beat entries that contain no information, it has no edge. This was the decisive instrument.

A note on provenance: the control-entry analysis behind the verdict was originally run in a scratch research environment; controls.py is a faithful reimplementation of those baselines against this repo's engine (the verdict tables are the original results).

Setup

python3 -m venv venv
venv/bin/pip install -r requirements.txt

Everything works with no API keys at all (Yahoo Finance fallback, ~60 days of 5-minute history). For longer histories — walk-forward wants ~180 days — add free Alpaca paper keys to .env (cp .env.example .env); a free Tradier sandbox token adds real option IV/spread for the costed runs.

Usage

# Backtest the watchlist on the underlying stock
venv/bin/python main.py backtest

# One ticker, simulated ATM option P&L with real IV + frictions, full trade log
venv/bin/python main.py backtest --ticker AAPL --costs --trades

# Current signal on each watchlist name
venv/bin/python main.py scan

# Grid-search parameters (worst-of train/test scoring)
venv/bin/python main.py optimize

# Anchored walk-forward OOS validation (needs multi-month history)
venv/bin/python main.py walkforward --source alpaca --costs

# Run the test suite (network-free)
venv/bin/python -m pytest

Running the control baselines against any frame:

import controls, marketdata
df = marketdata.intraday("AAPL", interval="5m", period="59d")
results = controls.run_controls(df, "AAPL")   # {name: stats}
for name, s in results.items():
    print(f"{name:>12}  PF {s['profit_factor']:.3f}  trades {s['trades']}")

Compare against the strategy's PF on the same frame: an entry with real alpha must clear every baseline by more than noise.

The strategy under test

Momentum/breakout on 5-minute bars: long when the close breaks the prior N-bar high on elevated volume (short the mirror image), with optional trend, opening-range, and volatility-regime filters. Exits: ATR stop (optionally trailing), ATR target, max-hold cap, and a hard end-of-day flatten — no overnight positions. Option mode "buys" an ATM contract and reprices it each bar with Black-Scholes. All parameters live in config.py as dataclasses; the optimiser sweeps them via dataclasses.replace without mutating global state.

Files

File	Purpose
backtest.py	Event-driven backtester (underlying + option modes, cost model, pluggable signal_fn)
controls.py	Signal-free control entries — the falsification baselines
walkforward.py	Anchored expanding-window out-of-sample validation
optimize.py	Two-stage grid search, worst-of train/test scoring
strategy.py	Momentum/breakout signal logic + ATR (no-look-ahead)
options_pricing.py	Black-Scholes pricing
option_data.py	Real ATM implied vol + bid/ask spread (best-effort, cached)
sizing.py	Affordability caps + risk-based contract sizing
market_regime.py	Broad-market trend gate (causal, shifted)
config.py	Every tunable as a dataclass; frictionless defaults reproduce
marketdata.py	Data layer — Alpaca IEX (real-time), Yahoo fallback
data.py	Yahoo chart API access (curl_cffi, TLS impersonation for 429 avoidance)
main.py	CLI: backtest / scan / optimize / walkforward
VERDICT.md	The negative result this repo exists to document
tests/	Network-free pytest suite (incl. the no-look-ahead regression test)

Testing

venv/bin/python -m pytest

The suite is deterministic and network-free (synthetic seeded frames), covers the no-look-ahead invariant, the backtester and its cost model, the walk-forward windowing, option-data fallback, sizing, and the control baselines, and runs in CI on every push (.github/workflows/tests.yml).

Disclaimer

This is research tooling, not financial advice. Its own conclusion is that the strategies it tested do not work; treat any PF > 1 you produce with it as guilty until proven innocent — that is what the controls are for.