Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Naming convention for backtesting methods #613

Open
KishManani opened this issue Dec 26, 2023 · 2 comments
Open

Naming convention for backtesting methods #613

KishManani opened this issue Dec 26, 2023 · 2 comments
Labels
documentation Improvements or additions to documentation

Comments

@KishManani
Copy link
Contributor

Hello! Loving the package so far. I was reading the backtesting documentation here. I think some of the naming conventions contradict what I've found in a review paper on forecasting evaluation. I was wondering whether there is wider disagreement on terminology in the field or whether this is potentially a typo in the documentation? Perhaps there is ambiguity in which origin we're referring to.

Here is an example. In the skforecast documentation the following is referred to as being fixed origin:

image

Whereas in the paper this setup is referred to as a rolling origin:
image

Thanks again!
Kishan

@JavierEscobarOrtiz
Copy link
Collaborator

Hello @KishManani

Thanks for opening the issue. If I understand correctly, it seems like the image on the right you provided with "rolling origin" is talking about the end of the training set or the origin of the validation set, "the forecast origin rolls forward and the forecast horizon is constant".

In the user guide, "fixed origin" and "rolling origin" refer to the origin of the training set (the green area in the gifs). Perhaps we can change it to "training set fixed origin" and "training set rolling origin". What do you think?

Best,

Javier

@JavierEscobarOrtiz JavierEscobarOrtiz added the documentation Improvements or additions to documentation label Dec 27, 2023
@KishManani
Copy link
Contributor Author

Hi Javier,

Thanks for the clarification. In my opinion "training set origin" is still ambiguous because it doesn't specify whether we're referring to the start or end of the training period.

Perhaps a sentence in the documentation clarifying which origin we're referring to (start of training, end of training, forecast start date or origin) would help.

Thanks,
Kishan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

2 participants