New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add common test to ensure all(predict(X[mask]) == predict(X)[mask]) #10420

Closed
jnothman opened this Issue Jan 8, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@jnothman
Member

jnothman commented Jan 8, 2018

I don't think we currently test that estimator predictions/transformations are invariant whether performed in batch or on subsets of a dataset. For some fitted estimator est, data X and any boolean mask mask of length X.shape[0], we need:

all(est.method(X[mask]) == est.method(X)[mask])

where method is any of {predict, predict_proba, decision_function, score_samples, transform}. Testing that predictions for individual samples match the predictions across the dataset might be sufficient. This should be added to common tests at sklearn/utils/estimator_checks.py

Indeed, #9174 reports that this is broken for one-vs-one classification. :'(

@Johayon

This comment has been minimized.

Show comment
Hide comment
@Johayon

Johayon Jan 8, 2018

Contributor

Hi, could I take this issue ?

Contributor

Johayon commented Jan 8, 2018

Hi, could I take this issue ?

@jnothman

This comment has been minimized.

Show comment
Hide comment
@jnothman

jnothman Jan 8, 2018

Member
Member

jnothman commented Jan 8, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment