AssertionError while using TrailingStrategy

[zlpatel]

I am seeing this error when try to run backtest with 10000 days worth data. It doesn't happen for all the stocks, but some. e.g AAPL, AMZN.

Index: 1946  Close: 35.88999938964844  ATR: 1.6433353276970901
Index: 1947  Close: 35.779998779296875  ATR: 1.5673829350797757
Index: 1948  Close: 35.779998779296875  ATR: 1.5232842084983242
Index: 1949  Close: 36.029998779296875  ATR: 1.4951925556138177
Index: 1950  Close: 36.13999938964844  ATR: 1.438393141851363
Index: 1951  Close: 35.65999984741211  ATR: 1.3906505347952947
Traceback (most recent call last):
  File "backtesting_usa.py", line 116, in <module>
    output = bt.run()
  File "/usr/local/lib/python3.8/site-packages/backtesting/backtesting.py", line 1165, in run
    strategy.next()
  File "backtesting_usa.py", line 83, in next
    super().next()
  File "/usr/local/lib/python3.8/site-packages/backtesting/lib.py", line 446, in next
    trade.sl = max(trade.sl or -np.inf,
  File "/usr/local/lib/python3.8/site-packages/backtesting/backtesting.py", line 633, in sl
    self.__set_contingent('sl', price)
  File "/usr/local/lib/python3.8/site-packages/backtesting/backtesting.py", line 652, in __set_contingent
    assert price is None or 0 < price < np.inf
AssertionError

I tried printing the index values for each iteration and I found that for this dataset, it processed till index 1951 and failed on 1952.

If I reduce the backtesting period to 5000, it works because it doesn't come across that piece of data (index 1952). I checked that data and verified the close price, and calculated ATR(14).

Below is the code snippet:

class Breakout(TrailingStrategy):
	timeperiod = 10
	position_size_decimal = 0.2
	min_close = []
	max_close = []
	def init(self):
		super().init()
		self.ema20 = self.I(EMA,self.data.Close,20,overlay=True)
		self.atr14 = self.I(ATR,self.data.High,self.data.Low,self.data.Close,14)
		self.set_atr_periods(14)
		self.set_trailing_sl(1)
		print(type(self.data.Close))
		self.min_close, self.max_close = MINMAX(self.data.Close, timeperiod=self.timeperiod)

	def next(self):
		super().next()
		index = len(self.data)-1
		print('Index: '+f'{index} '+' Close: '+f'{self.data.Close[index]} '+' ATR: '+f'{self.atr14[index]}')
		if not self.position.is_long and self.min_close[index] > (self.max_close[index] * 0.98) and self.max_close[index] < (self.min_close[index] * 1.02):
			#print('BUY ZONE:'+ f'{self.data.index[index]} {self.rsi2[index]} {self.rsi5[index]} {self.ema200[index]} {self.data.Close[index]}')
			self.buy(size=self.position_size_decimal)

I download data using yfinance python api. Any help will be greatly appreciated.

Originally posted by @zlpatel in #315

[kernc]

These are the culprit lines in TrailingStrategy:

backtesting.py/backtesting/lib.py

Lines 446 to 447 in 0b2325f

	trade.sl = max(trade.sl or -np.inf,
	self.data.Close[-1] - self.__atr[-1] * self.__n_atr)

The way max() can produce a value for which assert price is None or 0 < price < np.inf fails is if trade.sl is unset (you don't pass initial self.buy(..., sl=)) and self.data.Close[-1] - self.__atr[-1] * self.__n_atr is negative.
Meaning, the ATR in $ is greater than the price in $.

Can you propose a fix?

[zlpatel]

@kernc I looked at that code earlier. Still not sure how this code would return negative?

I changed the ATR period to 20 and sl to 1.5 ATR. Now it fails on Index 1439 for AMZN. I tried printing the ATR value and Close value for index 1439. Values are as below.

Index: 1438 Close: 22.049999237060547 High: 22.049999237060547 Low: 22.049999237060547 ATR: 0.8737164653968182
Index: 1439 Close: 22.170000076293945 High: 22.5 Low: 22.030000686645508 ATR: 0.853530607794702
In this case, self.data.Close[-1] - self.__atr[-1] * self.__n_atr

22.05 - (0.87*1.5) = 20.745 which is a positive value.

The only difference is that I use ta-lib ATR function while your api uses internal implementation to calculate ATR:

backtesting.py/backtesting/lib.py

Lines 425 to 433 in 0b2325f

	def set_atr_periods(self, periods: int = 100):
	"""
	Set the lookback period for computing ATR. The default value
	of 100 ensures a _stable_ ATR.
	"""
	h, l, c_prev = self.data.High, self.data.Low, pd.Series(self.data.Close).shift(1)
	tr = np.max([h - l, (c_prev - h).abs(), (c_prev - l).abs()], axis=0)
	atr = pd.Series(tr).rolling(periods).mean().bfill().values
	self.__atr = atr

Am I missing something here?

[kernc]

🤔 Ideally, someone would fire up a debugger and place a breakpoint here, then see what's what, and why the assertion fails.
Can you provide a piece of your data to try on?

[zlpatel]

@kernc Apologies. I am not quite familiar with python debugging. reading about debugging using pdb. But here's the dataset that I am using for my test. I will try to debug it.
AMZN.csv

[zlpatel]

@kernc so I started the debugger. price value is coming out as -70.88 and hence the AssertionError. I am still checking the price calculation.

[kernc]

You can always inject a little helper code:

if price < 0:  # our special condition of interest
    import ipdb; ipdb.set_trace()

Then you can do an up, up to go from the assertion up the stack to the TrailingStrategy.next() where all the fun is.

[zlpatel]

@kernc I have found the culprit. It's self.__atr[-1]

/usr/local/lib/python3.8/site-packages/backtesting/lib.py(433)set_atr_periods()
-> self.__atr = atr
(Pdb) pp atr[-1]
62.03802490234375

/usr/local/lib/python3.8/site-packages/backtesting/lib.py(440)set_trailing_sl()
-> self.__n_atr = n_atr
(Pdb) pp n_atr
1.5

/usr/local/lib/python3.8/site-packages/backtesting/lib.py(446)next()
-> trade.sl = max(trade.sl or -np.inf,
(Pdb) pp self.data.Close[-1]
22.170000076293945

so per the formula: trade.sl = max(trade.sl or -np.inf, self.data.Close[-1] - self.__atr[-1] * self.__n_atr)

22.17 - (62.04*1.5) = -70.89

the code shouldn't be using atr[-1]. It should be using atr[index] where index = len(self.data)-1

Because, when self.data in init() is different from self.data in next(). self.data in init() has all the data. while the one in next() gets accumulated data over iterations. i.e. 5 iterations -> 5 data. n iteration -> n data. so doing self.data.Close[-1] in next() gets you the last record. but the same is not true for self.__atr, because self.__atr always has full computed set from init(), so you need to use absolute index while accessing atr value from self.__atr.

Is my understanding correct?

[zlpatel]

@kernc do you want me to make changes and submit a merge request?

[kernc]

Excellent work!

Your proposed fix sounds good. If you'd like to submit a PR, that would be splendid! Don't forget a small, contrived unit/regression test.

[zlpatel]

@kernc thanks. do I need to have write permission to this repo?

[zlpatel]

@kernc oh nevermind. it says it will create a new branch.

[zlpatel]

@kernc I ran into this with some other data. Below are the values I printed from code. It fails on Index 84. Because 0.375 - (0.20375*2) = -0.0325 Notice atr values for past 2 indexes as well. May be we should look into how ATR is being calculated?

Index: 82 Open: 0.4375 High: 0.5 Low: 0.375 Close: 0.4375
Index: 82 atr: 0.20375
Index: 83 Open: 0.5 High: 0.5 Low: 0.4375 Close: 0.5
Index: 83 atr: 0.20375
Index: 84 Open: 0.4375 High: 0.5 Low: 0.375 Close: 0.375
Index: 84 atr: 0.20375

Here's how atr array looks like:

[ 0.20375 0.20375 0.20375 ... 126.44834717 126.07144531
125.39134521]

May be somewhere values are getting rounded off in ATR calculation as they might be too close to each other?

Below is the code I am using

`class DemaCross10_25(SignalStrategy,TrailingStrategy):
n1 = 10
n2 = 25

def init(self):
    # In init() and in next() it is important to call the
    # super method to properly initialize the parent classes
    super().init()
    
    # Precompute the two moving averages
    dema1 = self.I(DEMA, self.data.Close, self.n1)
    dema2 = self.I(DEMA, self.data.Close, self.n2)
    
    # Where dema1 crosses dema2 upwards. Diff gives us [-1,0, *1*]
    signal = (pd.Series(dema1) > dema2).astype(int).diff().fillna(0)
    signal = signal.replace(-1, 0)  # Upwards/long only
    
    # Use 95% of available liquidity (at the time) on each order.
    # (Leaving a value of 1. would instead buy a single share.)
    entry_size = signal * .2
            
    # Set order entry sizes using the method provided by 
    # `SignalStrategy`. See the docs.
    self.set_signal(entry_size=entry_size)
    
    # Set trailing stop-loss to 2x ATR using
    # the method provided by `TrailingStrategy`
    self.set_trailing_sl(2)`

Attaching data here if you want to test it.
NVR.csv