Optimizing Rolling Window Indicators

[G8XSU]

Regarding most of the rolling window indicators for example SMA,

Why isn't the previously calculated SMA Indicator used as an optimization similar to dynamic programming?

For e.g in SMA didn't we only need to exclude the oldest period value in average and include the latest period value to calculate one additional point in SMA ?

I see that most of the indicators are like this. Performance gain by this should be huge.
Am i missing something ?

[codeplea]

I don't think I'm understanding what you're asking.

For e.g in SMA didn't we only need to exclude the oldest period value in average and include the latest period value to calculate one additional point in SMA ?

That is exactly how the code works. For each output, it subtracts the oldest bar in the average, and adds in a new bar. See here.

Performance gain by this should be huge.

Performance is a goal of this library. There is a benchmark against ta-lib here.

[G8XSU]

If we were storing/using the previously calculate SMA, it would be O(1) implementation.
Similar to https://github.com/askmike/gekko/blob/develop/strategies/indicators/SMA.js

[codeplea]

It does run in linear time. I guess you didn't look at the code I linked to?

[G8XSU]

Hi,
I did look at the code, from what i understood, we are iterating over the input as fresh input in "for loop".
This is linear time implementation but this can be done in constant time if we have already computed SMA for previous window. (Previous SMA just didn't include the latest candle)
Is there some other platform where we can chat?

[G8XSU]

I think whats missing here is context,
If we calculate SMA the way we are currently doing we are kind of assuming that we have a single chart of candlesticks and then we computed the SMA for it just once and displayed it as a overlay over original chart.

But in reality, the candlestick graph keeps on updating and adding new candles.
This is true for :

1. Showing the graph live on some trading exchange.
2. Using trading bot in live market, which needs to calculate this indicator again and again for every new candle.
3. While testing existing trading-bot over historical data, we would need to calculate this indicator again and again since candles are being ingested one-at-a-time.

When we are adding new candle to graph every minute, it is sub-optimal to calculate whole SMA again and again every minute.
Instead we need to re-use the SMA which was computed without considering the latest candle from input. This will result in savings of redundant computation.

[codeplea]

I see. You're not talking about the initial calculation (in which linear time is optimal), but instead about updating the calculation over time.

We're working on a streaming interface for the next release that does exactly what you're asking. It's currently in the 0.9 branch. See here: https://github.com/TulipCharts/tulipindicators/blob/0.9/indicators/sma.c

[G8XSU]

Went through very useful discussion on #57 . I read through the comments and it was very insightful.

Is 0.9 #35 ready ? Why isn't this merged to mainline?

I will see if i can contribute to this project but i am not that familiar with writing optimized C code.
I am pretty new to technical analysis, i will try if i can implement some reference indicator.

I will go on and request support for streaming interface in tulipy.

[adrianog]

And... What happened?

[codeplea]

We lost the sponsor for this feature, and it's not a priority for me at this time. If someone else wants to sponsor the work, it'll happen much faster.

The way I use this library is primarily for back-testing, and the routines are already optimal for that. The calculation load for running live algos, where the implementation isn't optimal, is minuscule compared to back-testing.