Added demo for prediction intervals - #3954
Conversation
Codecov Report
@@ Coverage Diff @@
## main #3954 +/- ##
=====================================
Coverage 99.7% 99.7%
=====================================
Files 347 347
Lines 36790 36790
=====================================
Hits 36670 36670
Misses 120 120 Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
bccdbd1 to
054dda3
Compare
eccabay
left a comment
eccabay
left a comment
There was a problem hiding this comment.
Looks great! Just a couple small notes.
| }, | ||
| "vscode": { | ||
| "interpreter": { | ||
| "hash": "fb5a3afe2d0dd7ad0fd9e1ff89de4e0e95804490c629f36065bf8d930a66d311" | ||
| } |
There was a problem hiding this comment.
We should really have an automation or at least a check to strip these out.
| "source": [ | ||
| "While predictions that are generated by EvalML pipelines aim to be accurate as possible, it is very rarely the case that future results are the exact same values as predicted. Prediction intervals can help to contextualize a prediction by showing the range a future prediction is expected to fall within a certain likelihood. \n", | ||
| "\n", | ||
| "Given a set of predictions and a **trained** EvalML pipeline, the prediction intervals for this set of predictions is generated by calling `get_prediction_intervals()` on the pipeline's estimator:" |
There was a problem hiding this comment.
This is an important callout, but could you rephrase it slightly to mention that we don't just need the trained pipeline, we specifically need the transformed, ready for prediction features?
I'm also very out of the loop on prediction intervals, why do we need the transformed features?
There was a problem hiding this comment.
Updated to clarify that only a fitted estimator is needed and that the example is using the fitted estimator in a trained pipeline.
We need the transformed features as we're calling get_prediction_intervals() from the estimator directly. It expects to take input in that has already been through the preprocessing steps of the pipeline. We currently don't have a way to get prediction intervals at the pipeline level.
86ed01c to
536ad7c
Compare
jeremyliweishih
left a comment
jeremyliweishih
left a comment
There was a problem hiding this comment.
This looks great! Thanks @christopherbunn
536ad7c to
cbff41a
Compare
3 participants
Also includes a demo for forecasting that plots the prediction intervals.