Validate holdout dataset passed to predict/predict_proba for time series #2804
Conversation
Codecov Report
@@ Coverage Diff @@
## main #2804 +/- ##
=======================================
+ Coverage 99.8% 99.8% +0.1%
=======================================
Files 297 297
Lines 27719 27754 +35
=======================================
+ Hits 27651 27686 +35
Misses 68 68
Continue to review full report at Codecov.
|
freddyaboulton
force-pushed
the
2732-validate-holdout-dataset-ts
branch
from
e9e3611
to
1059466
Compare
freddyaboulton
requested review from
chukarsten,
bchen1116,
dsherry,
angela97lin,
christopherbunn,
eccabay,
jeremyliweishih and
ParthivNaresh
| X (pd.DataFrame): Data of shape [n_samples, n_features]. | ||
| objective (Object or string): The objective to use to make predictions. | ||
| X_train (pd.DataFrame): Training data. | ||
| y_train (pd.Series): Training labels. |
There was a problem hiding this comment.
Docstring doesn't match the function sig!
| y_holdout = self._create_empty_series(y_train) | ||
| X, y_holdout = self._convert_to_woodwork(X, y_holdout) | ||
| y_holdout = infer_feature_types(y_holdout) | ||
| self._validate_holdout_datasets(X, X_train) |
There was a problem hiding this comment.
Did you intend to validate the holdout datasets twice?
There was a problem hiding this comment.
My mistake lol
| }, | ||
| ) | ||
| X, y = ts_data | ||
| X_train, y_train = X.iloc[:15], y.iloc[:15] |
There was a problem hiding this comment.
might be good to just put the 15 into a variable in case we come back to change this for some reason or another down the road.
| elif thing_thats_wrong == "not-separated-by-gap": | ||
| X = X.iloc[15 + gap + 2 : 15 + gap + 2 + forecast_horizon] | ||
| else: | ||
| X = X.iloc[15 + gap + 2 : 15 + gap + 2 + forecast_horizon + 1] |
There was a problem hiding this comment.
Remind me again what the right length looks like here? What subsetting of X passes the test?
There was a problem hiding this comment.
Great question. X.iloc[15 + gap: 15 + gap + forecast_horizon]
freddyaboulton
force-pushed
the
2732-validate-holdout-dataset-ts
branch
from
a17c87a
to
7ade293
Compare
Pull Request Description
Fixes #2732
After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of
docs/source/release_notes.rstto include this pull request by adding :pr:123.