Repository navigation
Backtest: strategy decisions on delayed data, matching against newer exchange state #5441
graceyangfan
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
The strategy should decide from the older market data it has received; when its order arrives, the exchange should match against its newer market state. How can we express these two views in a Nautilus backtest?
This is a request to clarify the modeling contract and the smallest useful next step, not a bug report or a request to change defaults. It connects the information/state question in #2612 with the command-arrival scheduling question in #5403.
The concrete case
Assume valid, ordered tick/book data on aligned clocks:
The desired behavior is: at 150ms, decide using the 100ms quote; at 175ms, match using the exchange state after the 160ms update. With a fixed 50ms data delay throughout this example:
The 140ms and 160ms updates reach the strategy only at 190ms and 210ms. The strategy therefore cannot use them in its decision, even though the exchange can already use them for matching. These are different market views on one simulation clock, not access to data from after order arrival.
The invariant is: exchange state includes valid market updates that occurred by time T; strategy-visible market state includes only updates delivered by T. They may coincide when the market has not changed. The model must allow them to differ when updates occur during data and command transit.
flowchart LR subgraph exchange["Exchange state"] Q["100ms: quote occurs\nask = 100"] --> M140["140ms: ask = 101"] M140 --> M160["160ms: ask = 102"] M160 --> MATCH["175ms: process arriving buy\ncurrent ask = 102"] end subgraph strategy["Strategy delivery and command path"] RECEIVE["150ms: sees the 100ms quote\nask = 100"] -->|"Compute: 5ms"| SEND["155ms: submit buy"] SEND -->|"Command delay: 20ms"| ARRIVE["175ms: buy arrives"] end Q -->|"Data delay: 50ms"| RECEIVE ARRIVE --> MATCHArrows show causal steps, not a time-proportional scale. The price example assumes a complete valid feed and sufficient executable liquidity; it does not reconstruct an unobserved price path.
What is already implemented
Source reference: upstream
developat7b3ed05d98.add_datasorts byts_init; the loop routes each datum to the exchange, then the data engine, at itsts_init.ts_initplus operation latency. #5403 discusses release at a later settlement point.InflightOrdersand receipt-aware contingent handling.Two design lessons matter: #2612 identified local/venue state and response-delay concerns in 2025; that is historical context, not a claim that nothing has since been implemented. In #4865's later review, pending-update/fill handling allowed synchronous settlement machinery to be removed while retaining the execution channel. Any next step should build on that final design.
One candidate: combine the delays
The target above is the requirement. Moving data delay onto commands is one possible approximation to assess against it.
For the example:
The arithmetic preserves arrival time. It preserves behavior only if early evaluation produces the same command and the same relevant subsequent state transitions. The venue must receive the same command trace under the same market replay and matching assumptions.
A fixed data delay is insufficient:
flowchart LR subgraph reference["Separate delivery"] R100["100ms: quote A occurs\nStrategy is flat; A not delivered"] --> R120["120ms: existing order fills\nReport delivered; now long"] R120 --> R150["150ms: receive A\nAlready long: no buy"] end subgraph approximation["Collapsed delay"] C100["100ms: evaluate A while flat\nQueue buy for 175ms"] --> C120["120ms: same fill report\nNow long"] C120 --> C175["175ms: extra buy arrives"] endThe existing order was already at the venue before 100ms in both models. No added rule retracts the queued buy. Delaying execution cannot repair a decision made from different state.
Independent per-message signals can satisfy the equivalence conditions. Feedback-dependent strategies, receive reordering across feeds, local submission/risk state, timers and fill-triggered hedges may not. A single static combined delay also cannot represent varying data delays or distinguish quote-triggered commands from timer-triggered ones. Adding a compute-delay scalar does not simulate a busy strategy thread or its input queue.
Possible ways forward
These are two separate decisions, rather than one large refactor:
A possible implementation uses one clock with exchange-market events, strategy-delivery events and command-arrival events. That is a candidate to assess, not a conclusion that two full caches or a replacement matching engine are necessary. Delayed execution reports are a separate scope: they require controlling when local order/position knowledge changes, not merely postponing a callback.
Boundaries: do not rewrite every data type to
ts_init = ts_event. Open-stamped completed bars would leak future OHLCV; book messages need sequence/atomic-batch preservation; vendor timestamps need interpretation. Exact arrival cannot recover missing market data.Questions for participants
Validation should compare commands as well as fills: an independent-signal control, the feedback counterexample, receive reordering, mixed command triggers, arrival between data points, and equal-time/batch boundaries. Current evidence is source inspection and standalone integer-time reasoning checks, not a tested Nautilus implementation.
Input from @stastnypremysl and @cjdsellers (#2612), @ypark66 (#5403), @mdou7 (#4891), and @frslvr (#5054) would be welcome, particularly corrections to these boundaries. The aim is to agree on the contract before proposing implementation work.
All reactions