Spec

Spec

What to run, and on which dates. Spec turns a spec file, or the arguments of sbt2.backtest, into resolved, hashed run specs, and owns the splits. It never runs anything; Run does.

spec.load(path, overrides) → list[ResolvedRunSpec]: parse → expand → resolve → validate → hash.

A resolved run spec maps one-to-one onto a nautilus BacktestRunConfig: nautilus's own constructor arguments kept as plain data, plus the few fields nautilus has no place for (split, part, strategy import path). sbt2 has no config model of its own, and no dataclass that copies nautilus config fields; one function builds the nautilus configs from the resolved spec, and run.execute hands them to nautilus. The plain form exists because nautilus's config objects can't be hashed, compared or serialized (see below).

The layout of that plain data belongs to spec. Code outside it asks the resolved spec instead of indexing its venue and data arguments: venue_name, currency (the starting balance's), liquidation_enabled, data_types (the nautilus classes the run streams), instrument_ids and data_window (the span the data is read over, warm-up included). Run's preflight builds its data selection from these.

Run spec

A run spec is the complete description of one backtest: everything needed to run it, and nothing else.

Question	Example
Which strategy?	my_strats.orb:Orb (an import path in a spec file, the class itself in Python; no registry)
With which parameters?	lookback = 20, stop_pct = 0.02
On which instruments?	BTCUSDT-LINEAR.BYBIT (a full nautilus InstrumentId)
Over which period?	period 2020-01-01 to 2025-01-01, split FractionSplit(validation=0.2, test=0.2), part train
On which simulated exchange?	venue profile bybit_linear
Capital, seed	capital = "10000 USDT" (nautilus money, becomes the venue's starting_balances), seed = 42
Equity sampling	equity_interval = "1h" (default)
Liquidation	liquidation = true or false; overrides the venue profile's liquidation_enabled
What bars are built from	bars = "trades" (default) or "candles"
Risk	risk = { max_notional_per_order = { "BTCUSDT-LINEAR.BYBIT" = 50000 }, max_order_submit_rate = "100/00:00:01", drawdown_limit = 0.2 } (optional)

• It contains no results, output paths, report options or storage locations. Anything that doesn't change the backtest stays out, so it can't change the hash. The benchmark is therefore not in the run spec: it's strategy metadata, overridable when generating a report (see Results).
• The equity interval is in the run spec, because it changes the output.
• liquidation is a venue argument, so it's hashed. Pre-flight refuses liquidation = true on a run that streams no quotes (see Run).
• bars names a bar source: a BarSource subclass that gives the data types the declared bars are built from and aggregated_from, the external bars they aggregate from (None for ticks). TradeBars ("trades") and CandleBars ("candles") are the built-in ones, in one registry in spec/bars, so a new source is one class and one registry entry. An unknown name fails with UnknownBarSourceError listing the known ones.
• bars changes the streamed data types, so it changes the hash. With "candles", only time bars of whole minutes on LAST prices can be built; any other declared bar (sub-minute, tick, volume or value bars, BID / ASK / MID) fails resolving with CandleBarError.
• risk holds nautilus's RiskEngineConfig fields under their own names, max_notional_per_order (a table keyed by instrument id), max_order_submit_rate and max_order_modify_rate, plus drawdown_limit, a fraction of peak equity strictly between 0 and 1 that the strategy base enforces (see Strategy). Any other key fails with UnknownRiskKeyError listing the valid ones; a limit outside (0, 1) fails with DrawdownLimitError.
• risk is hashed only when set: a run spec without it hashes exactly as it did before risk existed.
• The seed has a fixed default, so random fill models stay reproducible. It goes into the fill model's random_seed, the only place nautilus takes one; without a fill model it changes nothing, not even the hash.

Spec files and batches

You write specs as TOML. A file may contain lists, and is then expanded into one concrete run spec per combination:

strategy    = "my_strats.orb:Orb"
instruments = ["BTCUSDT-LINEAR.BYBIT", "ETHUSDT-LINEAR.BYBIT"]    # one universe
period      = [2020-01-01, 2025-01-01]
split       = { validation = 0.2, test = 0.2 }
venue       = "bybit_linear"
capital     = "10000 USDT"
                                     # no part: train and validation, × 2
[params]
lookback = [20, 50]                  # × 2 → 4 run specs
stop_pct = 0.02

• What expands: a list in [params], and part. Every list expands into its Cartesian product with every other. instruments is the strategy's universe, not a list to expand: one run trades all of them. period is a pair of dates.
• part is a part's name, a list of names (part = ["validation", "test"]), or missing, which means train and validation. Test runs only when asked for.
• Order: by part first, then by the [params] lists, with keys in file order and values in list order. The same file always expands in the same order.
• All or nothing. Every run is resolved and validated before anything runs, so one bad combination fails the whole load. Its typed error carries a note naming the combination (in the train run with bar='1-SECOND-LAST').
• No duplicates. Two combinations that resolve to the same hash, such as lookback = [20, 20] or stop_pct = ["0.02", 0.02], fail with DuplicateRunError. An empty list, which would expand into no runs, fails with EmptyListError.
• Overrides replace top-level keys before expansion, so a list in an overridden params expands too.

Studies

study = "ma-cross-btc", at the top of a spec file, puts its runs in a study: a named collection of runs of one strategy, on one fixed context, that differ only in their parameters (see Results).

• The name is a slug of lowercase letters, digits and dashes ([a-z0-9-]+), since it names a folder; anything else fails with InvalidStudyNameError.
• Each resolved run carries its study as a Study: the name, and the context the study fixes, which is every spec-file key but params, part and study (strategy, instruments, period, split, venue, capital, seed, equity interval, liquidation, bars, risk), with the defaults filled in and the split as its document, as JSON.
• The study is not hashed: like the source, it is neither in the hashed document nor compared, so the same backtest has the same hash in or out of a study.
• A spec without study runs outside any study.

Whether a run fits its study is checked by batch, before anything runs (see Run).

With an [optimise] table (optimiser = "grid", objective = "sharpe"), the optimiser searches the strategy's declared search space instead, tuning on train and ranking on validation, and lists in [params] override it for their parameter. See Library.

Resolve and hash

Resolving replaces every name with its contents, using nautilus's own argument names:

• venue profile → a BacktestVenueConfig, plus the asset profile for the nautilus asset_class and instrument_class it names;
• instruments → nautilus InstrumentId with InstrumentId.from_str; spec files carry full ids because nautilus has no function that builds venue-specific ids from a bare symbol, and an instrument whose venue isn't the venue profile's name fails;
• split and part → the part's dates, which become the BacktestRunConfig start / end;
• warm-up → from the strategy's warmup(params); the data configs' start_time is the part's start minus warm-up;
• declared inputs → one BacktestDataConfig per data type: bars → trades, or with bars = "candles" → Bar with each instrument's 1-minute candle bar type in bar_types. The strategy base subscribes to the bar type built from each declared BarSpecification (see Strategy);
• equity interval → PortfolioConfig.snapshot_interval_ms, with use_mark_prices from the asset class (the asset profile's portfolio_config);
• parameters → defaults filled in and validated against the strategy by the strategy API's resolve_params, then placed in the strategy's ImportableStrategyConfig;
• risk → its nautilus fields become the engine's RiskEngineConfig as they are; drawdown_limit goes to the strategy as StrategyRun.drawdown_limit, never to the risk engine (see Strategy). Both appear under risk in the hashed document, which has no risk key when neither is set.

Spec imports the strategy here for metadata only (parameter schema, defaults, warm-up). The resolved run spec is hashed, so the same hash always means the same backtest, even if a venue profile changes.

The hash is SHA-256 over the resolved spec as canonical JSON (sorted keys, enums by name, decimals as strings). Machine-only fields (catalog path, chunk_size), the venue profile's source and the run's study are never part of it: they say where data is read or fetched from, not what the backtest is. v2's config objects have no JSON form, can't be pickled, and two identical configs don't compare equal; BacktestRunConfig.id is a random UUID. So nautilus can't hash it for us.

ResolvedRunSpec.from_document(document) is the inverse of to_json: it rebuilds the spec a stored run's spec.json holds, with the same hash, typing the values back (instrument ids, data types, dates, the split, the asset profile, the strategy's parameters through its schema, the drawdown limit) and putting the asset profile's venue defaults back as their enum objects. aggregated_from is not in the document; a run that streams Bar data runs on candles, so it is read back from the data types. The source and the study are not in the document either: a rebuilt spec has neither, and neither takes part in comparing specs. The strategy must be importable, as for resolving. A run stored before model kinds, whose venue names a model by import path, fails with OutdatedRunError ("predates model kinds"); its path is never imported or translated.

Unknown or mistyped strategy parameters fail before anything runs, with a message listing the valid ones. Strategies are referenced by import path, so there's no registry to maintain and names can't silently collide. v1 had a duplicate registry key that made one strategy unreachable.

Venue profiles

A named venue profile is a table in the config folder's venues.toml (see Architecture); load(path, venue_profiles, overrides=None) takes that file, with no default, and a missing one fails with MissingConfigError, naming it. sbt2-core ships none: the GUI holds the defaults. A profile is BacktestVenueConfig's own keyword arguments written in TOML, plus three sbt2 keys: the asset profile, named with nautilus's own enums asset_class (e.g. CRYPTOCURRENCY) and instrument_class (e.g. SWAP), and source, the data source that serves its instruments, from which Run fetches missing data. The keyword arguments go straight to the constructor: it already accepts enum names as strings (oms_type = "NETTING"), and it rejects bad values and unknown fields, so sbt2 has no venue schema, enum mapping or validation of its own. Only the asset profile's keys are converted, with nautilus's AssetClass.from_str and InstrumentClass.from_str.

Some arguments take Python objects, not plain values: fee_model, fill_model, latency_model, margin_model and modules (a list). The TOML names each one by a kind, from a closed set sbt2 supports, plus the keyword arguments of its model: { kind = "...", config = { ... } }. A profile never names a model by import path, so nothing that reads a profile imports a module the profile names. spec/resolve/models.py holds one StrEnum of kinds per argument, mapped to its nautilus model by an explicit registry; sbt2.core.spec exports the enums.

Argument	Kinds	Nautilus model
fee_model	FeeModelKind.MAKER_TAKER (maker_taker)	MakerTakerFeeModel(maker_rate, taker_rate)
	FeeModelKind.FIXED (fixed)	FixedFeeModel(commission), the commission as a money string such as "1 USDT"
fill_model	FillModelKind.DEFAULT (default)	DefaultFillModel(prob_fill_on_limit, prob_slippage); the seed goes into random_seed
latency_model, margin_model, modules	none yet

Resolving a profile builds each model once from its kind and config, as a check, and keeps the {kind, config} table as plain data; the run config builds them again. So a profile fails on load, before anything runs: a kind its argument doesn't have fails with UnknownModelKindError, listing the known ones (known: none for an argument without kinds); a config its model doesn't take fails with InvalidModelConfigError, naming the argument, the kind and nautilus's message; a model table without a kind, such as one naming an import path, fails with InvalidVenueProfileError.

[bybit_linear]
name              = "BYBIT"
oms_type          = "NETTING"
account_type      = "MARGIN"
asset_class       = "CRYPTOCURRENCY"
instrument_class  = "SWAP"
source            = "bybit"
fee_model         = { kind = "maker_taker", config = { maker_rate = "0.0002", taker_rate = "0.00055" } }

It selects nautilus built-ins, each model through its kind:

Concern	Nautilus built-in
Fees	maker_taker or fixed. Required: v2 rejects a venue without an explicit fee model, even a zero-fee one.
Fills	default, nautilus's probabilistic DefaultFillModel. Other nautilus fill models join the enum as new kinds. With L2 data, market orders walk the real book.
Latency	none yet; nautilus's default of no latency
Margin	none yet; nautilus's default margin model, or a cash account
Liquidation	venue config (liquidation_enabled, liquidation_trigger_ratio, liquidation_cancel_open_orders), where the asset class has it; a spec's liquidation key overrides liquidation_enabled
Carry	perp funding is settled by the venue itself from streamed FundingRateUpdates; modules holds a carry module, by its kind, only for an asset class whose carry the venue doesn't settle
Account	account type, settlement currency, netting or hedging, leverage, starting balance

Spec resolves the profile into the venue config; Run passes it straight to BacktestNode. See Asset classes for carry.

Rules carried over from v1's mistakes:

• Slippage is never folded into fees.
• Bar strategies fill at the close of the bar that produced the signal or later, never at a price from inside it.
• Instruments are real specs, never guessed.

Splits

A split divides a run spec's period into three parts, in time order: train, where the optimiser tunes; validation, where it ranks candidates and where you check what was tuned, as often as you like; and test, the final answer, which is locked (see Run). A run covers one part. A spec file or backtest call without part expands into a train and a validation run; test must be asked for.

Splits are Splitter objects, part of the run spec, not named in config. Every run spec has one: a run over the whole period would cover the test dates without passing the test lock. A Splitter gives the parts of a period (parts(period)) and describes itself as plain data (document(), its kind and arguments), which is what is hashed. Splitter is an abstract base class: it owns parts and document, and each splitter only says where validation and test start. A spec file's split table becomes the splitter whose arguments are exactly its keys:

Form	Parts
FractionSplit(validation=0.2, test=0.2) (spec file: split = { validation = 0.2, test = 0.2 })	test is the last 20% of the period, validation the 20% before it, train the rest; each boundary rounded to a whole UTC day
DateSplit(validation_start="2023-01-01", test_start="2024-01-01") (spec file: split = { validation_start = 2023-01-01, test_start = 2024-01-01 })	the same by date; each date is floored to its UTC day

• Both document themselves as kind = "Split", so the hash doesn't depend on the class name.
• Resolving turns the split into the part's dates, and the dates are hashed. A fraction of an explicit period never moves when more data arrives; v1's fraction of the loaded data did.
• UTC, half-open [start, end). Each part starts where the one before it ends: there is no embargo. Only the boundaries between parts are rounded; the period's own ends are kept.
• Each part is run separately. Each starts flat with fresh capital. Metrics cover only the part's own dates.
• Warm-up may read data from before the part (validation's warm-up reads the end of train). That's not leakage: nothing is traded, fitted or measured on it.

Errors

Typed errors, all subclasses of SpecError (a ValueError): unknown spec key (listing the valid ones), unknown risk key (UnknownRiskKeyError, listing the valid ones), a drawdown limit outside (0, 1) (DrawdownLimitError), unknown bars value (UnknownBarSourceError), a declared bar candles can't build (CandleBarError), unknown parameter (the strategy API's UnknownParameterError), unknown split kind (a split table whose keys fit no splitter, such as fractions mixed with dates or half of either, UnknownSplitError), missing split, unknown part, two runs of a spec file that resolve to the same backtest (DuplicateRunError), an empty list (EmptyListError), a missing venue profiles file (MissingConfigError), unknown venue profile, venue profile without a source or with a model named without a kind (InvalidVenueProfileError), unknown model kind (UnknownModelKindError, listing the known ones), a model config its model doesn't take (InvalidModelConfigError), a stored run that predates model kinds (OutdatedRunError), fractions that aren't positive, add up to 1 or more, or leave a part empty once rounded, dates outside the period or out of order, instrument on another venue.