Skip to content

Updated pipeline.get_prediction_intervals() to add trend prediction interval information from STL decomposer - #4093

Merged
christopherbunn merged 13 commits into
mainfrom
TML-6964_stl_pred_intervals
Mar 23, 2023
Merged

Updated pipeline.get_prediction_intervals() to add trend prediction interval information from STL decomposer#4093
christopherbunn merged 13 commits into
mainfrom
TML-6964_stl_pred_intervals

Conversation

@christopherbunn

@christopherbunn christopherbunn commented Mar 20, 2023

Copy link
Copy Markdown
Contributor

Resolves #4060

@codecov

codecov Bot commented Mar 20, 2023

Copy link
Copy Markdown

Codecov Report

Merging #4093 (483a613) into main (6fd5daf) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #4093     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        349     349             
  Lines      37588   37644     +56     
=======================================
+ Hits       37469   37525     +56     
  Misses       119     119             
Impacted Files Coverage Δ
...nents/transformers/preprocessing/stl_decomposer.py 100.0% <100.0%> (ø)
...valml/pipelines/time_series_regression_pipeline.py 100.0% <100.0%> (ø)
...nent_tests/decomposer_tests/test_stl_decomposer.py 100.0% <100.0%> (ø)
.../tests/pipeline_tests/test_time_series_pipeline.py 100.0% <100.0%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@christopherbunn
christopherbunn force-pushed the TML-6964_stl_pred_intervals branch from d72d719 to b8f784b Compare March 21, 2023 19:14
Comment thread evalml/pipelines/components/transformers/preprocessing/stl_decomposer.py Outdated
@christopherbunn
christopherbunn force-pushed the TML-6964_stl_pred_intervals branch 2 times, most recently from be57e09 to 4be75cb Compare March 22, 2023 17:22
@christopherbunn christopherbunn changed the title Tml 6964 stl pred intervals Updated pipeline prediction intervals to add trend prediction interval information from STL decomposer Mar 22, 2023
@christopherbunn christopherbunn changed the title Updated pipeline prediction intervals to add trend prediction interval information from STL decomposer Updated pipeline.get_prediction_intervals() to add trend prediction interval information from STL decomposer Mar 22, 2023
@christopherbunn
christopherbunn marked this pull request as ready for review March 22, 2023 20:47

@eccabay eccabay left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good! Just a few comments about code conciseness and expected behavior.

Comment thread evalml/pipelines/components/transformers/preprocessing/stl_decomposer.py Outdated
Comment thread evalml/pipelines/components/transformers/preprocessing/stl_decomposer.py Outdated
Comment thread evalml/pipelines/time_series_regression_pipeline.py
Comment thread evalml/pipelines/time_series_regression_pipeline.py Outdated

@chukarsten chukarsten left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for refactoring some of that code out of _project_trend(), definitely looks more concise. Left a few comments to address, but looking forward to getting this out!

Comment on lines +405 to +406
if coverage is None:
coverage = [0.95]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dumb question, and I'm guilty of doing this too, why not just have this be the default value in the function def?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread evalml/pipelines/time_series_regression_pipeline.py
@christopherbunn
christopherbunn force-pushed the TML-6964_stl_pred_intervals branch from 8719001 to 4acaa2b Compare March 23, 2023 14:30
@christopherbunn
christopherbunn enabled auto-merge (squash) March 23, 2023 15:49
Comment on lines +213 to +215
has_stl = STLDecomposer.name in list(
self.component_graph.component_instances.keys(),
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, I suggested we use STLDecomposer.name, but now on the heels of Jeremy's work I feel like we should follow his lead and check for any(isinstance(c, STLDecomposer) for c in self.component_graph.component_instances.values() instead 😅 just to be safe

stl.fit(X_train, y_train)

def assert_pred_interval_coverage(pred_interval):
expected_coverage = [0.95] if coverage is None else coverage

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check here that there aren't any other cover values that got added somehow? i.e. when expected_coverage = [0.95] we only have the two columns?

@christopherbunn
christopherbunn merged commit eb621d9 into main Mar 23, 2023
@christopherbunn
christopherbunn deleted the TML-6964_stl_pred_intervals branch March 23, 2023 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pipeline.inverse_transform() is not ran when determining prediction intervals for decomposer pipelines

3 participants