ADVSTEX with limit entry orders?

[Quanatee]

Great package, been trying it for about a month and I've found it to be very flexible!

I've been using ADVSTEX and there doesn't seem to be an obvious way to define limit entry orders, and subsequently, filled entries. The class seems to assume that entries are filled order entries or market bought.

The entries I have built for ADVSTEX are signal based entries, but I would like to move towards the above direction or being able to reflect limit order entries that are filled or canceled (time-based).

Briefly, I think I can rudimentarily get around this by running the entries through a custom indicator to reflect only filled entry order, but that may not easily account for orders that are never filled. It seems that this package can benefit from an integrated function within ADVSTEX itself. Would like to hear your thoughts on this.

[polakowo]

The ADVSTEX generator a strict exit generator and has no logic to manipulate entries - those have to be prepared and provided by the user.

There are several options you can go: the first one is creation of a custom entry & exit generator (as you did apparently) with a custom entry function that does some basic order management and ADVSTEX exit function, the second one is filtering entries beforehand using signals accessor (but this assumes that your entries do not depend upon previous exits).

If you can elaborate more on limit entry orders (are those stop entry orders that are canceled once a stop is reached or simple buy stops?) and provide me with an example on how you currently handle them, I can help integrating a more general version of it into the package.

[Quanatee]

Sorry for the delayed response. I wanted to work on this issue as soon as your reply but was caught up in other aspects. To answer your question, this issue relates to simple limit buy orders.

Implementing a limit buy order has a few considerations:

1. The limit price may never hit, and thus the limit order never filled;
2. If the limit order never fills and the signal changes, the unfilled order is stale and should be canceled.
3. [Open ended] How long should a limit buy stay open before cancelling?

I wrote a simple example to implement in my system that fulfils the above considerations by running the assumption (on point 3) that consecutive signals belong to the same "signal session", and therefore unfilled limit orders are stale and should be canceled if they are left unfilled when the signal changes.

It seems though that a time-based expiry setting for limit orders are the convention so you may factor that into your development. "Signal session" style works for me, with the side benefit that the possibility of overtrading of the same signal won't happen in this style.

import numpy as np
import pandas as pd

index = pd.date_range('1/1/2000', periods=20, freq='T')
d = {
    'Signal': np.random.randint(2, size=20),
    'Limit Entry Price': np.random.uniform(8, 9, size=20),
    'Close': np.random.uniform(8.4, 8.6, size=20),
}
bt_df = pd.DataFrame(d, index=index)
# Make Limit Entry Prices persist across consecutive rows where Signals are True
# We must cancel unfilled orders if Signal changes in live
bt_df.loc[(bt_df['Signal'].shift() == bt_df['Signal']), 'Limit Entry Price'] = np.nan
bt_df['Limit Entry Price'] = bt_df['Limit Entry Price'].ffill()
bt_df.loc[bt_df['Signal'] == 0, 'Limit Entry Price'] = np.nan
bt_df['Limit Entry'] = False
bt_df.loc[(bt_df['Signal'] == True) & (bt_df['Limit Entry Price'] > bt_df['Close']), 'Limit Entry'] = True
bt_df.loc[(bt_df['Limit Entry'].shift() == bt_df['Limit Entry']) & (bt_df['Limit Entry'] == True), 'Limit Entry'] = False

print(bt_df)

# I then run ADVSTEX to get stop exits
# advstex = vbt.ADVSTEX.run(
        #     bt_df['Limit Entry'], bt_df['Open'], bt_df['High'], bt_df['Low'], bt_df['Close'],
        #     hit_price=bt_df['Limit Entry Price'],
        #     sl_stop=sl_stops, tp_stop=tp_stops,
        #     param_product=True
        # )

# Result
                        Signal  Limit Entry Price     Close  Limit Entry
2000-01-01 00:00:00       0                NaN  8.564140        False
2000-01-01 00:01:00       0                NaN  8.520179        False
2000-01-01 00:02:00       1           8.757558  8.470297         True
2000-01-01 00:03:00       0                NaN  8.412795        False
2000-01-01 00:04:00       1           8.238643  8.505277        False
2000-01-01 00:05:00       0                NaN  8.404852        False
2000-01-01 00:06:00       1           8.022066  8.498470        False
2000-01-01 00:07:00       0                NaN  8.471920        False
2000-01-01 00:08:00       1           8.226325  8.438089        False
2000-01-01 00:09:00       1           8.226325  8.439872        False
2000-01-01 00:10:00       1           8.226325  8.502265        False
2000-01-01 00:11:00       1           8.226325  8.501408        False
2000-01-01 00:12:00       1           8.226325  8.517973        False
2000-01-01 00:13:00       1           8.226325  8.421095        False
2000-01-01 00:14:00       0                NaN  8.446641        False
2000-01-01 00:15:00       1           8.584325  8.502960         True
2000-01-01 00:16:00       1           8.584325  8.560458        False
2000-01-01 00:17:00       1           8.584325  8.451963        False
2000-01-01 00:18:00       0                NaN  8.534686        False
2000-01-01 00:19:00       0                NaN  8.410393        False

On a side note, I saw that shorting function has been added to portfolio. Hope that ADVSTEX (and signals modules) will include support for it!

[polakowo]

Buy limit orders can be done in multiple ways in vectorbt, but all of them involve setting relative stops in percentage terms because vectorbt puts focus on hyperparameter optimization. For example, 1% stop:

entry_signals = bt_df['Signal'].astype(bool)\
    .vbt.signals.first()\
    .vbt.signals.generate_stop_exits(bt_df['Close'], -0.01, exit_wait=0)

Here we use generate_stop_exits to generate entries, those can then be fed to 1) ADVSTEX that preserves your entries and tries to find exits for each of them, or 2) IADVSTEX that tries to fill the first entry order -> if filled, finds an exit -> if found, finds the next entry order after that exit -> etc..

A better approach in your case would involve working directly with Portfolio.from_order_func that processes timestamps one by one (streaming approach), so you can define your own order management logic there, instead of relying upon multiple layers of signal generation that can expose you to look-ahead bias, etc. I personally use signal generators only for analyzing the distribution of signals and other data analyses, while backtesting is done iteratively.

[Quanatee]

I do use relative stops in percentages so that isn't a problem for me.

That's a novel solution to generate entries (using generate_stop_exits). I think I can use it. Can you explain a bit what vbt.signals.first() does?

Regarding generate_stop_exits, does the entry_wait parameter imply that there is no need to manually shift the prices/signals to avoid look-ahead bias?

[polakowo]

vbt.signals.first selects the first True out of a series of consecutive True values. There is also a possibility to select the first True out of a series delimited by another boolean array (reset_by argument).

The only drawback of generate_stop_exits: it doesn't fill the hit price, so you need to do it yourself by using forward fill. The more advanced method that takes OHLC generate_adv_stop_exits also fills the hit price and the type of a stop being hit (as you can define multiple stop types at different timestamps).

Regarding exit_wait: vectorbt uses an approach, where for each entry signal, it defines a range after it and searches an exit signal within this range. exit_wait defines where this range starts. exit_wait = 0 means you can start searching for exit signals from the same bar as the entry. Normally, exit_wait is 1 because vectorbt allows only placing one order at a timestamp, but I used 0 just to be consistent with your example.