Preserve input indices on estimators and pipelines' predict/predict_proba/transform/inverse_transform

[angela97lin]

Closes #1639

[codecov]

Codecov Report

Merging #2979 (c8c20dc) into main (a65e026) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #2979     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        312     312             
  Lines      29893   29979     +86     
=======================================
+ Hits       29802   29888     +86     
  Misses        91      91             

Impacted Files	Coverage Δ
...tanding/prediction_explanations/_user_interface.py	100.0% <ø> (ø)
...ents/estimators/classifiers/lightgbm_classifier.py	100.0% <ø> (ø)
...nents/estimators/classifiers/xgboost_classifier.py	100.0% <ø> (ø)
evalml/pipelines/classification_pipeline.py	100.0% <100.0%> (ø)
...ents/estimators/classifiers/catboost_classifier.py	100.0% <100.0%> (ø)
...valml/pipelines/components/estimators/estimator.py	100.0% <100.0%> (ø)
...ponents/estimators/regressors/prophet_regressor.py	100.0% <100.0%> (ø)
.../components/transformers/encoders/label_encoder.py	100.0% <100.0%> (ø)
...ents/transformers/preprocessing/log_transformer.py	100.0% <100.0%> (ø)
...transformers/preprocessing/polynomial_detrender.py	97.7% <100.0%> (ø)
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a65e026...c8c20dc. Read the comment docs.

[eccabay]

Why do we need this line? It looks like we only use it to grab the index. Is it just in case the input y is a numpy array?

[angela97lin]

Yeah, not really related to index work, just standardizing since we do so for other components :')

[freddyaboulton]

@angela97lin Thank you this looks great! I think there's a bug in the test cause LightGBM.predict is not preserving the index?

[freddyaboulton]

Thanks for chasing this down!

[freddyaboulton]

Do we need this line given that the LabelEncoder preserves the index?

[angela97lin]

Theoretically if all of the components properly handle preserving indices including the label encoder then no... but I figured it would be good to have on the pipelines just in case!

[freddyaboulton]

Nice thanks for adding this!

[freddyaboulton]

Why doesn't this test catch this?

import pandas as pd
from evalml.demos import load_breast_cancer
from evalml.pipelines.components import LightGBMClassifier

X, y = load_breast_cancer()
lgbm = LightGBMClassifier()
X.index = range(25, 25 + X.shape[0]
lgbm.fit(X, y)

pd.testing.assert_index_equal(lgbm.predict(X).index, X.index)

AssertionError: Index are different

Index values are different (100.0 %)
[left]:  RangeIndex(start=0, stop=569, step=1)
[right]: RangeIndex(start=25, stop=594, step=1)

predict_proba does preserve the index though

[angela97lin]

Wow, this is a really good catch! Tracking this down, it's because we have separate logic for LightGBM and XGBoost's predict methods depending on whether or not we need to encode the targets (they internally have label encoders). Since my test uses the X_y_binary fixture, it doesn't use the label encoder. In the case where we do use the label encoder, the label encoder's inverse_transform wipes the index so we have to set it again:

        X, _ = super()._manage_woodwork(X)
        X.ww.set_types(self._convert_bool_to_int(X))
        X = _rename_column_names_to_numeric(X, flatten_tuples=False)
        predictions = super().predict(X)
        if not self._label_encoder:
            return predictions
        predictions = pd.Series(
            self._label_encoder.inverse_transform(predictions.astype(np.int64)),
            index=predictions.index,
        )
        return infer_feature_types(predictions)

Adding another set of tests for numeric vs non-numeric targets 😅

[bchen1116]

LGTM! Glad we're now able to preserve the index in our estimators and pipelines

[freddyaboulton]

Thank you @angela97lin ! Looks good to me!

[chukarsten]

Angela, this looks great and I think it greatly enhances our test coverage with respect to index preservation. I don't see anything that blocks, but I am curious whether we should do any index preservation within infer_feature_types? I know that would cut out a few lines, from the individual pipeline classes, and it also seems like something we should expect of the function. If not, also, nbd.

[chukarsten]

Is part of the underlying problem here that infer_feature_types() should be preserving the index? Should we do it in there?

[angela97lin]

@chukarsten Unfortunately, I think regardless of our infer_feature_types behavior (which I think should preserve indices but I'm not positive), the issue is that a lot of third party libraries might return data such as np.arrays or new pandas objects that don't have the indices attached to them. 🥲

[chukarsten]

I know we do this in similar tests, but it's bold. I'm always hesitant to see pseudo-randomness added into a testing suite. I think it opens the door for potential flakiness and I'm not sure about the implications across platforms. I'm pretty sure this usage is fine, though.

[angela97lin]

Heh, I had taken inspiration from our other tests but agreed! The pseudo-randomness isn't really necessary here so I'll remove it from this instance and our other tests that use this :)