Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multiple TimeSeries for historical_forecasts(), residuals() and backtest() + automatic start #1318

Merged
merged 29 commits into from
Dec 20, 2022

Conversation

dumjax
Copy link
Contributor

@dumjax dumjax commented Oct 26, 2022

Summary

Add the possibility to historical forecast, backtest and compute residuals on multiple time series.

Also automatically detects the starting point if start is not specified

@dumjax dumjax changed the title init multiple TimeSeries for historical_forecasts(), residuals() and backtest() + automatic start Oct 26, 2022
@maximilianreimer
Copy link

Thanks for the great work! Is there a apporximate time line for when this is going to be part of a realease?

@dumjax dumjax marked this pull request as ready for review November 11, 2022 12:49
@codecov-commenter
Copy link

codecov-commenter commented Nov 26, 2022

Codecov Report

Attention: Patch coverage is 96.42857% with 7 lines in your changes missing coverage. Please review.

Project coverage is 93.81%. Comparing base (0d924c8) to head (b1c6291).
Report is 359 commits behind head on master.

Files with missing lines Patch % Lines
darts/utils/utils.py 88.00% 3 Missing ⚠️
darts/models/forecasting/forecasting_model.py 98.43% 2 Missing ⚠️
...arts/models/forecasting/torch_forecasting_model.py 90.90% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           master    #1318    +/-   ##
========================================
  Coverage   93.81%   93.81%            
========================================
  Files          94       94            
  Lines        9436     9544   +108     
========================================
+ Hits         8852     8954   +102     
- Misses        584      590     +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@hrzn hrzn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great @dumjax, I only put minor comments.
Maybe the main one is a proposal to use a generator instead of eagerly filling up a list. I think this could scale better and avoid blocking calls until all results are computed. However, I think it could be done in a later PR, as this could maybe be done in other parts of Darts as well.
The second fairly important comment would be managing the progress tracking of the outer loop.

def extreme_lags(
self,
) -> Tuple[
Union[int, None],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually find it a bit clearer to write Optional[int] instead of Union[int, None].

darts/models/forecasting/forecasting_model.py Show resolved Hide resolved
darts/models/forecasting/forecasting_model.py Show resolved Hide resolved
darts/models/forecasting/forecasting_model.py Outdated Show resolved Hide resolved
darts/models/forecasting/forecasting_model.py Outdated Show resolved Hide resolved
@@ -554,13 +903,14 @@ def backtest(
] = metrics.mape,
reduction: Union[Callable[[np.ndarray], float], None] = np.mean,
verbose: bool = False,
) -> Union[float, List[float]]:
) -> Union[float, List[float], Sequence[float], List[Sequence[float]]]:

"""Compute error values that the model would have produced when
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments as for historical_forecasts() docstring

darts/models/forecasting/forecasting_model.py Show resolved Hide resolved
docs/source/conf.py Show resolved Hide resolved
dumjax and others added 6 commits November 30, 2022 16:08
Co-authored-by: Julien Herzen <julien@unit8.co>
Co-authored-by: Julien Herzen <julien@unit8.co>
Co-authored-by: Julien Herzen <julien@unit8.co>
Co-authored-by: Julien Herzen <julien@unit8.co>
Co-authored-by: Julien Herzen <julien@unit8.co>
Co-authored-by: Julien Herzen <julien@unit8.co>
Copy link
Contributor

@hrzn hrzn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one @dumjax 👏
I have made a few changes directly in your branch. I think we can merge...

@hrzn hrzn merged commit d315dd8 into master Dec 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants