Best pattern for expensive precomputed indicators while preserving backtest/live parity?

[davedleo]

Hi everyone,

I’m trying to understand the recommended NautilusTrader pattern for a strategy that uses a computationally expensive indicator/signal, where:

• in live trading, the indicator needs to be calculated incrementally from the incoming market-data stream;
• in backtests, calculating it incrementally for every historical event is prohibitively expensive, so I would like to precompute it offline and feed the resulting values into the backtest;
• nevertheless, I want to keep exactly the same strategy logic and event ordering between backtest and live trading, so that I don’t introduce a separate “backtest strategy” implementation.

For example, suppose the strategy consumes bars:

bar N arrives
    ↓
indicator/signal for bar N becomes available
    ↓
strategy.on_bar(bar N)
    ↓
strategy may submit an order

In live trading, I understand that registering an indicator with the relevant bar type allows NautilusTrader to update the indicator before on_bar() is invoked.
My question is how to implement the equivalent pattern when the indicator has been precomputed as historical data.

Ideally I would have something conceptually like:

Historical/backtest:

precomputed_signal[N] ──┐
                        ├──> signal available ──> on_bar(N)
bar[N] ─────────────────┘

Live:

incoming bar[N]
      ↓
indicator.update(bar[N])
      ↓
signal available
      ↓
on_bar(N)

The important requirement is that the strategy must never observe bar[N] before the corresponding signal for N is available, and conversely the signal must not accidentally be one event behind.

So, my questions are:

1. Is there a recommended/native NautilusTrader pattern for this?
2. Should the precomputed signal be represented as:
   • a custom Data type,
   • an Indicator,
   • something in the DataCache,
   • or another mechanism?
3. How should event ordering be handled?
   For example, if the signal is derived from bar N, how can I guarantee that the precomputed signal for N is available before on_bar(bar_N) executes?
4. Is there a recommended way to make the same strategy code work in both environments, e.g.:

def on_bar(self, bar):
    signal = self.signal.value
    ...

where self.signal is fed incrementally in live trading but sourced from precomputed historical values during backtesting?
5. More generally, what is considered the idiomatic NautilusTrader architecture for maintaining backtest/live parity when a computation is too expensive to perform repeatedly during a large backtest?

Thanks in advance!

[mkzung]

1 and 2. I would make the signal a custom data type with the customdataclass decorator, not an indicator. Live, an actor updates the indicator on each bar and publishes signal N with bar N's timestamps. In the backtest there is no actor, you add the precomputed signals as data.

3. Add the bars first and the signals second, in two add_data calls. On equal ts_init the stream added first goes first, so bar N always comes before signal N:

   nautilus_trader/crates/backtest/src/data_iterator.rs

   Lines 135 to 144 in f95360d

   	impl Ord for HeapEntry {
   	fn cmp(&self, other: &Self) -> std::cmp::Ordering {
   	// min-heap on replay key, then priority sign (+/-) then index
   	self.key
   	.cmp(&other.key)
   	.then_with(|| self.priority.cmp(&other.priority))
   	.then_with(|| self.index.cmp(&other.index))
   	.reverse() // BinaryHeap is max by default -> reverse for min behavior
   	}
   	}

   
   Live, the data engine puts a bar in the cache before it publishes it, so the order is the same.

4. Yes, if the strategy decides in on_data when signal N arrives, not in on_bar. Live, the signal is computed from bar N, so on_bar(N) can run before it exists. In on_data read bar N from self.cache.bar(bar_type) and skip if its ts_event is not the signal's. That catches a signal one bar behind.

5. Build the offline series with the same indicator code the actor runs, then run a short window both ways and compare the values.