You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Backtest: a latency-delayed command waits for the instrument's next datum, so a bursty feed inflates the effective latency
#5403
Prepared with AI assistance and reviewed before posting.
This is a design question rather than a bug. The behaviour below follows the settlement table in docs/concepts/backtesting/execution-flow.md ("Latency-delayed commands"), but its effect on bursty real tick data is large, so I'd like to discuss it. Moved here from #5402.
What a user would expect
With StaticLatencyModel(base_latency_nanos=30_000_000), a command should reach the simulated venue
about 30 ms after it is sent, and match against the book as it is at that moment.
What happens
The command becomes due at ts_init + latency, but nothing settles it at that time. It waits for
the next settlement point: the instrument's next datum, a non-snapshot timer, or the shutdown drain.
With a feed that arrives in bursts, the effective latency becomes the gap to the next burst.
In the reproducer (quotes 500 ms apart, ask rising 2 ticks per quote), a marketable IOC is accepted 500 ms after submit. By then the book has moved, so the IOC is canceled. If the strategy sets a
no-op time alert at submit + 30 ms, the same order is accepted and filled at 30 ms.
On real data (Interactive Brokers tick-by-tick L1 quotes, which arrive in bursts about 0.5 s apart),
a 30.6 ms latency became 490–590 ms. An IOC that filled live was canceled in the backtest. With a
one-shot timer at each command's due time, the backtest reproduced the orders and fills of two live
sessions exactly.
Where it happens
rc6 is the v2.0.0rc6 tag (it matches the PyPI sdist). develop is 39adb53a28. The settlement
logic is the same in both.
What
rc6
develop
Arrival time is ts_init + latency (generate_inflight_command)
crates/backtest/src/exchange.rs L856–L870
L901–L915
A due command is released only if matches_scope: same-timestamp, or the current datum's instrument
Both protect against filling across a gap against an old book. On a dense but bursty feed the
trade-off goes the other way: at the due time the book is only a few hundred milliseconds old, and
waiting for the next burst matches the order against a book from up to about 500 ms after it
should have arrived.
Why a user-side workaround is incomplete
A strategy timer works (see the reproducer), and so does an actor that scans in-flight orders in the
cache and sets an alert at each due time. But the actor has to infer commands from order events,
and some commands leave none:
cancel_all_orders(strategy_only=False) sends CancelAllOrders without an OrderPendingCancel
(rc6 crates/trading/src/strategy/mod.rs L941–L956).
A second modify_order while the order is PENDING_UPDATE adds no OrderPendingUpdate
(mark_order_pending_update, L776–L778).
A command sent from a strategy's own timer or from on_start is only found if a datum or a scan
falls before its due time.
These still settle at the next settlement point. The engine knows every arrival time already
(max_inflight_command_ts), so it can settle them exactly.
Possible options (for your judgement)
Settle at the due time when the instrument's book is fresh. The engine would wake at the
earliest in-flight arrival, as it already does at shutdown, and release that command if the
instrument has had data within some bound. Otherwise it would keep the current behaviour, which
preserves Latency-delayed MARKET order fills at a data-less UTC midnight at the stale last price #5054's protection.
Documentation only. State in "Latency-delayed commands" that the effective delay is the
latency or the time to the next settlement point for the instrument, whichever is longer. Then
mention the explicit-timer workaround.
Steps to reproduce
pip install nautilus_trader==2.0.0rc6 (or a develop wheel) and save the script below as repro_latency_settlement.py.
Run python repro_latency_settlement.py. The IOC is accepted and canceled at +500 ms.
Run python repro_latency_settlement.py --wake-at-due-time. The IOC is accepted and filled at
+30 ms.
Code snippets or logs
"""Latency-delayed order commands settle at the next datum, not at their due time.One instrument, three quotes 500 ms apart (the ask rises 2 ticks per quote), and a30 ms StaticLatencyModel base latency. The strategy submits an IOC BUY limited atthe first quote's ask, so it is marketable for 500 ms. With``--wake-at-due-time`` it also sets a no-op time alert at submit + 30 ms, whichmakes the engine settle the command at its due time."""importsysimportnautilus_traderfromnautilus_trader.backtestimportBacktestEngine, BacktestEngineConfigfromnautilus_trader.executionimportDefaultFillModel, FixedFeeModel, StaticLatencyModelfromnautilus_trader.modelimport (
AccountType,
BookType,
Currency,
Equity,
InstrumentId,
Money,
OmsType,
OrderSide,
Price,
Quantity,
QuoteTick,
StrategyId,
Symbol,
TimeInForce,
TraderId,
Venue,
)
fromnautilus_trader.tradingimportStrategy, StrategyConfigINSTRUMENT_ID=InstrumentId.from_str("AAPL.XNAS")
T0_NS=1_772_064_000_000_000_000GAP_NS=500_000_000# bursty feed: next quote 500 ms laterLATENCY_NS=30_000_000# 30 ms base latencyWAKE_AT_DUE_TIME="--wake-at-due-time"insys.argvclassIocOnFirstQuote(Strategy):
def__init__(self, config: StrategyConfig) ->None:
super().__init__(config)
self.submitted=Falsedefon_start(self) ->None:
self.subscribe_quotes(INSTRUMENT_ID)
defon_quote(self, quote: QuoteTick) ->None:
ifself.submitted:
returnself.submitted=Trueorder=self.order_factory.limit(
INSTRUMENT_ID,
OrderSide.BUY,
Quantity.from_int(1),
quote.ask_price, # marketabletime_in_force=TimeInForce.IOC,
)
self.submit_order(order)
ifWAKE_AT_DUE_TIME:
self.clock.set_time_alert_ns(
"wake", self.clock.timestamp_ns() +LATENCY_NS, callback=lambda_: None
)
engine=BacktestEngine(BacktestEngineConfig(trader_id=TraderId("TESTER-001"), bypass_logging=True))
engine.add_venue(
Venue("XNAS"),
OmsType.NETTING,
AccountType.CASH,
[Money.from_str("1000000 USD")],
fill_model=DefaultFillModel(1.0, 0.0, 42),
fee_model=FixedFeeModel(Money.from_str("0 USD")),
latency_model=StaticLatencyModel(LATENCY_NS, 0, 0, 0),
book_type=BookType.L1_MBP,
)
engine.add_instrument(
Equity(INSTRUMENT_ID, Symbol("AAPL"), Currency.from_str("USD"), 2, Price.from_str("0.01"), 0, 0)
)
engine.add_data(
[
QuoteTick(
INSTRUMENT_ID,
Price.from_str(f"{100.00+0.02*i:.2f}"), # market moves up 2 ticks per quotePrice.from_str(f"{100.01+0.02*i:.2f}"),
Quantity.from_int(100),
Quantity.from_int(100),
T0_NS+i*GAP_NS,
T0_NS+i*GAP_NS,
)
foriinrange(3)
]
)
engine.add_strategy(IocOnFirstQuote(StrategyConfig(strategy_id=StrategyId("REPRO-001"))))
engine.run()
print(f"nautilus_trader {nautilus_trader.__version__}, wake_at_due_time={WAKE_AT_DUE_TIME}")
print(f"latency {LATENCY_NS/1e6:.1f} ms, quote gap {GAP_NS/1e6:.1f} ms")
(order,) =engine.cache.orders()
submit_ns=Noneforeventinorder.events():
ifsubmit_nsisNoneandtype(event).__name__=="OrderSubmitted":
submit_ns=event.ts_eventifsubmit_nsisnotNone:
print(f" {type(event).__name__:<16} +{(event.ts_event-submit_ns) /1e6:8.3f} ms")
print(f" final status: {order.status}")
engine.dispose()
Output, with engine log lines removed (the same on rc6 and on 2.0.0rc8.dev20261021+19673):
nautilus_trader 2.0.0rc6, wake_at_due_time=False
latency 30.0 ms, quote gap 500.0 ms
OrderSubmitted + 0.000 ms
OrderAccepted + 500.000 ms
OrderCanceled + 500.000 ms
final status: CANCELED
nautilus_trader 2.0.0rc6, wake_at_due_time=True
latency 30.0 ms, quote gap 500.0 ms
OrderSubmitted + 0.000 ms
OrderAccepted + 30.000 ms
OrderFilled + 30.000 ms
final status: FILLED
nautilus_trader 2.0.0rc8.dev20261021+19673, wake_at_due_time=False
latency 30.0 ms, quote gap 500.0 ms
OrderSubmitted + 0.000 ms
OrderAccepted + 500.000 ms
OrderCanceled + 500.000 ms
final status: CANCELED
nautilus_trader 2.0.0rc8.dev20261021+19673, wake_at_due_time=True
latency 30.0 ms, quote gap 500.0 ms
OrderSubmitted + 0.000 ms
OrderAccepted + 30.000 ms
OrderFilled + 30.000 ms
final status: FILLED
Related: #4891, #5054 (and the follow-up comment there on the shutdown drain), #5253 (per-row
settlement of in-flight orders within one book event).
Specifications
OS platform: Linux-6.12.76-linuxkit-x86_64-with-glibc2.36 (Docker, linux/amd64)
Python version: 3.13.13
nautilus_trader version: 2.0.0rc6, and 2.0.0rc8.dev20261021+19673
Installed from: the PyPI wheel (rc6) and the package index wheel from packages.nautechsystems.io (develop)
Adapter/venue (if applicable): none in the reproducer (backtest only). The real-data observation used Interactive Brokers tick-by-tick L1 quotes.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
This is a design question rather than a bug. The behaviour below follows the settlement table in
docs/concepts/backtesting/execution-flow.md("Latency-delayed commands"), but its effect on bursty real tick data is large, so I'd like to discuss it. Moved here from #5402.What a user would expect
With
StaticLatencyModel(base_latency_nanos=30_000_000), a command should reach the simulated venueabout 30 ms after it is sent, and match against the book as it is at that moment.
What happens
The command becomes due at
ts_init + latency, but nothing settles it at that time. It waits forthe next settlement point: the instrument's next datum, a non-snapshot timer, or the shutdown drain.
With a feed that arrives in bursts, the effective latency becomes the gap to the next burst.
In the reproducer (quotes 500 ms apart, ask rising 2 ticks per quote), a marketable IOC is accepted
500 ms after submit. By then the book has moved, so the IOC is canceled. If the strategy sets a
no-op time alert at submit + 30 ms, the same order is accepted and filled at 30 ms.
On real data (Interactive Brokers tick-by-tick L1 quotes, which arrive in bursts about 0.5 s apart),
a 30.6 ms latency became 490–590 ms. An IOC that filled live was canceled in the backtest. With a
one-shot timer at each command's due time, the backtest reproduced the orders and fills of two live
sessions exactly.
Where it happens
rc6 is the
v2.0.0rc6tag (it matches the PyPI sdist). develop is39adb53a28. The settlementlogic is the same in both.
ts_init + latency(generate_inflight_command)crates/backtest/src/exchange.rsL856–L870matches_scope: same-timestamp, or the current datum's instrumentexchange.rsL94–L102, L748–L763,process_commandsL1596–L1628ts_init, scoped to its instrumentcrates/backtest/src/engine.rsL924–L942,settlement_scopeL975–L998Allengine.rsL1805–L1822engine.rsL1084–L1097So the run loop never wakes at an in-flight arrival time during the run. It does at shutdown.
The design tension
I understand why it works this way, and I'm not asking for the default to change:
instrument's data cannot release an order against stale state. The "fill at the next execution
datum via insert latency" pattern described there depends on this.
order does not fill at a stale price across a data gap such as a weekend.
Both protect against filling across a gap against an old book. On a dense but bursty feed the
trade-off goes the other way: at the due time the book is only a few hundred milliseconds old, and
waiting for the next burst matches the order against a book from up to about 500 ms after it
should have arrived.
Why a user-side workaround is incomplete
A strategy timer works (see the reproducer), and so does an actor that scans in-flight orders in the
cache and sets an alert at each due time. But the actor has to infer commands from order events,
and some commands leave none:
cancel_all_orders(strategy_only=False)sendsCancelAllOrderswithout anOrderPendingCancel(rc6
crates/trading/src/strategy/mod.rsL941–L956).modify_orderwhile the order isPENDING_UPDATEadds noOrderPendingUpdate(
mark_order_pending_update, L776–L778).on_startis only found if a datum or a scanfalls before its due time.
These still settle at the next settlement point. The engine knows every arrival time already
(
max_inflight_command_ts), so it can settle them exactly.Possible options (for your judgement)
earliest in-flight arrival, as it already does at shutdown, and release that command if the
instrument has had data within some bound. Otherwise it would keep the current behaviour, which
preserves Latency-delayed MARKET order fills at a data-less UTC midnight at the stale last price #5054's protection.
would stay as it is, so MARKET order fills at wrong price when a second instrument is registered before it, despite LatencyModel #4891's pattern is unaffected.
latency or the time to the next settlement point for the instrument, whichever is longer. Then
mention the explicit-timer workaround.
Steps to reproduce
pip install nautilus_trader==2.0.0rc6(or a develop wheel) and save the script below asrepro_latency_settlement.py.python repro_latency_settlement.py. The IOC is accepted and canceled at +500 ms.python repro_latency_settlement.py --wake-at-due-time. The IOC is accepted and filled at+30 ms.
Code snippets or logs
Output, with engine log lines removed (the same on rc6 and on
2.0.0rc8.dev20261021+19673):Related: #4891, #5054 (and the follow-up comment there on the shutdown drain), #5253 (per-row
settlement of in-flight orders within one book event).
Specifications
nautilus_traderversion: 2.0.0rc6, and 2.0.0rc8.dev20261021+19673packages.nautechsystems.io(develop)All reactions