question: chapter 15, regression, the calculation of return

[js20181010]

chapter 15, regression section, in the book, it is:
In [28]: data['strat_ols_1'] = data['pos_ols_1'] * data['returns']
In [29]: data['strat_ols_2'] = data['pos_ols_2'] * data['returns']

should we use shift(1) as same as in section "Vectorized Backtesting" (cell 15)?
In [28]: data['strat_ols_1'] = data['pos_ols_1'].shift(1) * data['returns']
In [29]: data['strat_ols_2'] = data['pos_ols_2'].shift(1) * data['returns']

(cell 15 is : In [15]: data['Strategy'] = data['Position'].shift(1) * data['Returns'] )

[yhilpisch]

No, a shift is not necessary in this instance since the regression is based on historical data (lags) only.

This is slightly different to the SMA example which uses the same day closing price to calculate the SMA and derive the signal.

[js20181010]

Thank you for your swift reply.
agree, the regression is based on lags only. <-- this is very clear.

my questionable point is the meaning of returns & subsequently the position
returns (label) is derived from EOD stock data. (my understanding it is a stock's close price) :
day n returns = log(day n close / day n-1 close)
day n returns is only available by end of the day.
the position (pos_ols_1) is derived from returns, it means pos_ols_1 is only available by end of the day as well.

 if using below for trading strategy return calculation with shift(1)
     data['strat_ols_1'] = data['pos_ols_1'].shift(1) * data['returns']
     it means: use position of "day n-1" to trade on "day n", then calculate "day n" strat_ols return
 
 however, if using below to calculate the strategy return for "day n", it seems its meaning is not clear:
     data['strat_ols_1'] = data['pos_ols_1'] * data['returns']

how about your thoughts?

[yhilpisch]

Whatever you do, it obviously needs to be consistent. The code as presented in the book is consistent. Your suggestion including the .shift(1) is not consistent if you stick to lagged data only as features. This is because you shift the position from n to n+1. Your lags are n-1, n-2, n-3, etc. The prediction is for n. With shift you move it to n+1. You need to multiply the day n prediction with day n return because you would enter the position at day n-1 (most recent lag).

…

On Thu, Mar 4, 2021 at 4:47 PM js20181010 ***@***.***> wrote: Thank you for your swift reply. agree, the regression is based on lags only. <-- this is very clear. my questionable point is the meaning of *returns* & subsequently the *position* returns (label) is derived from EOD stock data. (my understanding it is a stock's close price) : day n returns = log(day n close / day n-1 close) day n returns is only available by end of the day. the position (pos_ols_1) is derived from returns, it means pos_ols_1 is only available by end of the day as well. **if using below for trading strategy return calculation with shift(1)** data['strat_ols_1'] = data['pos_ols_1'].shift(1) * data['returns'] **_it means: use position value of "day n-1" to trade on "day n", then calculate "day n" strat_ols return_** **however, if using below to calculate the strategy return for "day n", it seems its meaning is not clear:** data['strat_ols_1'] = data['pos_ols_1'] * data['returns'] how about your thoughts? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABC4LADHNRZBP2QFBVPKT63TB6TQBANCNFSM4YPIQFSA> .

-- Dr Yves J Hilpisch CEO | http://tpq.io | @dyjh <http://twitter.com/dyjh> The Python Quants | The AI Machine

[js20181010]

Thanks for your detail explanation.
It has cleared my doubt.

the prediction of day n is based on data of day n-1, n-2. --> the prediction for day n is available by end of day n-1.
so we could make trade decision for day n on "eve" of day n-1 according to this prediction.

thank you very much once more to clear my misunderstanding.