Abstract module usage examples #43

Open
v3nz3n opened this Issue Jul 17, 2013 · 6 comments

Comments

Projects
None yet
3 participants

v3nz3n commented Jul 17, 2013

I've had a look at the module documentation for TA-Lib as well as the abstract-specific guide, yet I am still unclear as to what exactly the abstract API can do for me (and how). Specifically, I would like to see a code example that instantiates a custom indicator which maintains indicator value state and periodically calculates, say, RSI from an individual input value, as opposed to an array of input values.

What I envision is the ability to pass values to an indicator sequentially (as they become available), so for example, instead of maintaining a numpy array of 700 items to calculate RSI on 5 minute candles - every 5 minutes - I wonder if it is possible to pass an abstract indicator function the candle close price, once every 5 minutes, and for it to output the 14-period RSI value calculated from state. This would be better suited to my application which keeps track of 9 different indicator values for 5 different timeframes indefinitely and around the clock.

Whilst numpy arrays are convenient for generating once-off indicator value arrays, a live system with ongoing indicator calculations would be more easily maintained and more memory efficient were the TA-Lib object to maintain indicator state somehow. Is this something the abstract API can do?

If not, I see an alternative in having a rotating deque (of 14 items - the RSI period) which can serve as input to the abstract indicator. A code example of custom data type implementation via TA-Lib abstract would be appreciated.

Contributor

briancappello commented Jul 18, 2013

Hi @v3nz3n! As it stands now, the abstract interface is only a relatively simple wrapper intended to provide a consistent API around all indicators in the function interface. It currently is not directly capable of the behaviour you describe: another generic wrapper would be needed on top of the abstract interface to manage the deque(s).

The underlying TA-Lib is written in C and, for every call, each indicator requires at the very least input array(s) at least one longer than that indicator's lookback window. From the returned results array, your wrapper would only care about the most recent value returned by the TA-Lib library. This is an example of how the Quantopian guys handle converting "streaming data" to the numpy arrays required by the underlying C.

So, as far as I understand the implementation details, I unfortunately don't think you'd actually gain anything by using a deque; ultimately, the data structure would still need conversion (copying) to be usable by C. Potentially this SO question for numpy deque-like behavior and the author's linked code might be helpful?

Best,

Contributor

briancappello commented Jul 19, 2013

@v3nz3n: I threw together this gist that shows the simple (read: naive) example outlined above with deque. It uses composition over inheritance; extend at will.

v3nz3n commented Jul 21, 2013

@briancappello: Great thanks for the two replies and example code. I have been traveling since you posted so have not had the connectivity to reply sooner. Glancing at the code it seems exactly what I had in mind and definitely not a solution I would have conjured by myself. As explained in my original post, the sheer bulk of simultaneous indicators I require at different timeframes, requires me to adapt what I am doing with respect to resources and management. A stateful and rotary solution seemed most logical and I am glad you concur.

In a nutshell, my app identifies indicator divergence from price at several candle timeframes as a means of determining trend termination. Hence, MACD, RSI and stochastics (all with Bollinger Bands) need to be calculated at 5, 15, 60 and 240min timeframes and then analysed for divergence. This is done for several instruments so the number of arrays quickly multiply. I have optimised where I can, but the most obvious resource-hog seems the TA-Lib need for arrays with sufficient lookback. For once-off indicator calculations this is ok, but for ongoing tick-level calculations it presents a leviathan! You understood my explanation and I am grateful for the code assistance.

Thanks again, and I will post my experience with implementation!

v3nz3n commented Jul 23, 2013

The code example is working great!

I am probably trying to cut too close to the bone here, but wonder if it is possible to pass the abstract function an array of, say, only the 'close' prices as opposed to the OHLC & V (which is four times more than what is actually needed)? I tried passing a single key dict {'close': array([ 11.53, ...])} as well as a regular array, but the errors talib returns are "an integer is expected" and "ambiguous truth value" respectively.

Edit: I just noticed that this is covered in the docs at http://mrjbq7.github.io/ta-lib/abstract.html as well as the inline docs via help(abstract.Function). There is both the function set_input_arrays() as well as the parameter price='x' to achieve custom and more compact input.

Owner

mrjbq7 commented Jul 24, 2013

@v3nz3n are you trying to call RSI with only the close price? I'm a little confused. You should only need to pass the required inputs, and you could always set them all to the same array value?

v3nz3n commented Jul 25, 2013

@mrjbq7 thanks for responding. Your advice will be appreciated:

I am trying to minimize the inputs to the abstract API Functions. I would like to be able to pass an array (with length = lookback +1) to the abstract function in the same manner that the regular talib function API takes a numpy array as input.

With the help of @briancappello who provided some example code I am able to use a deque as a fixed length datastore which (when the abstract Fn put() method is called) converts the OHLC & V data stored in the deque to the expected dictionary of arrays. Since RSI (for example) requires only a single array (e.g. the 'close' price) I have adapted the conversion process to create a dictionary with empty arrays/lists for redundant keys:

{ 'high': [], 'close': [1,2,3...], 'open': [], 'low': [], 'volume': [] }

This works but I would prefer to simply provide an array of, say, the 'close' prices. I had a look at the 'set_input_arrays' function and incorporated this as follows, but clearly the integration is incomplete:

class MyFunction(object):
    def __init__(self, fn_name, *args, **kwargs):
        self._fn = Function(fn_name, *args, **kwargs)
        self._maxlen = self._fn.lookback + 1
        self._q = deque(maxlen=self._maxlen)

    def set_input_arrays(self, input_data):
        if Function.set_input_arrays(self, input_data):
            return True
        elif isinstance(input_data, np.ndarray):
            input_arrays = Function.get_input_arrays(self)
            # convert input_data to input_arrays and then call the super
            Function.set_input_arrays(self, input_arrays)
            return True

    def put(self, bar):
        self._q.append(bar)
        if len(self._q) < self._maxlen:
            return None
        else:
            #d = {'open': np.asarray([b.open for b in self._q]),
            #     'high': np.asarray([b.high for b in self._q]),
            #     'low': np.asarray([b.low for b in self._q]),
            #     'close': np.asarray([b.close for b in self._q]),
            #     'volume': np.asarray([b.volume for b in self._q]),
            #     }
            d = np.asarray([b.close for b in self._q])
            self._fn.input_arrays = d
            results = self._fn(d)
            if isinstance(results, (list, tuple)):
                return tuple([r[-1] for r in results])
            else:
                return results[-1]

How can the above custom abstract class accept a single array and assume it is the required 'close' input?

I apologize for asking the developers here for assistance with this trivial matter. Any pointers will be appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment