Lessons from v1

Lessons from v1

A review of sbt, the reason v2 exists.

Results that couldn't be fully trusted

• Funding was left out of PnL. It was tracked on the side, so Sharpe, PnL and every optimizer objective ignored it.
• Slippage was a fixed fee, computed from the first close and never updated as prices moved, then capped.
• Instruments were made up. Ticks were guessed from price differences; size precision, max quantity and margin were fixed.
• Strategies tracked their own position state, updated on submit rather than on fill, so it diverged when orders were rejected.
• No real stop or limit orders. Stops were emulated in on_bar and exited at the close.
• 252-day annualization for a market that trades 365 days.
• The L2 runner silently fell back to the first catalog instrument when the requested one wasn't found.
• A duplicate registry key made one strategy unreachable.
• Walk-forward: no warm-up on out-of-sample windows, no portfolio support, averaged Sharpes, and a "maximize trade count" objective.
• Portfolios: rebalances could use stale prices depending on insertion order, and plugins received every instrument's bars.

Architecture

• One runner class did everything: data discovery, instrument synthesis, cost modelling, engine assembly, execution, stats, artifact spilling, database writes and printing.
• It fought nautilus instead of using it: custom feather files with metadata encoded in filenames, hand-rolled indicators that grew forever, and two separate data systems.
• Three parallel strategy base classes with duplicated sizing and order code.
• A single god-object config mixing every concern, with parameters from three sources.
• "Plugin" meant two unrelated things.
• The results schema was tied to one validation method, and dead scheduler code was never removed.
• Docs drifted from the code, and research material (PDFs, agent files) lived in the repo.

Worth keeping

• An explicit data input so tests run headless.
• Failing loudly on unknown parameters.
• The warm-up gating idea (active_from).
• The research notes on how nautilus handles multiple instruments.