Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] add _pairwise property to BaseSearchCV #13925

Closed
wants to merge 7 commits into from

Conversation

@isrobson
Copy link

isrobson commented May 22, 2019

Fixes #13920

Previously BaseSearchCV did not support a 'precomputed' distance metric.

This PR simply adds a _pairwise property to the BaseSearchCV to ensure that some X evaluated with the 'precomputed' distance metric can be properly split during cross-validation by _safe_split

@isrobson isrobson changed the title add _pairwise property to BaseSearchCV [MRG] add _pairwise property to BaseSearchCV May 22, 2019
Copy link
Member

jnothman left a comment

Please add a test

@jnothman

This comment has been minimized.

Copy link
Member

jnothman commented on sklearn/model_selection/tests/test_search.py in 2546ef9 May 23, 2019

when the attribute name is const, you should just use est._pairwise = ...

This comment has been minimized.

Copy link
Author

isrobson replied May 23, 2019

Yes, that would be a bit simpler

@jnothman

This comment has been minimized.

Copy link
Member

jnothman commented on sklearn/model_selection/tests/test_search.py in 2546ef9 May 23, 2019

verity -> verify

Copy link
Member

jnothman left a comment

Maybe what I really want here is a non-regression test for the example posted in the original issue.

setattr(est, '_pairwise', _pairwise_setting)
cv = PairwiseCV(est)

# check if cv is pairwise

This comment has been minimized.

Copy link
@jnothman

jnothman May 23, 2019

Member

this comment is redundant with the variable name

cv = PairwiseCV(est)

# check if cv is pairwise
cv_is_pairwise = getattr(cv, '_pairwise', False)

This comment has been minimized.

Copy link
@jnothman

jnothman May 23, 2019

Member

why not just use cv._pairwise?

This comment has been minimized.

Copy link
@isrobson

isrobson May 23, 2019

Author

I was following the convention of safe_split but hit the 80 character limit so I broke it into a separate line. I suppose since checking cv._pairwise won't throw an error for not being instantiated, we can just use it here. I'll make the swap

@isrobson

This comment has been minimized.

Copy link
Author

isrobson commented May 24, 2019

I made your suggested edits and added a classification test similar to #13920 with KNeighborsClassifier, although I used the make_classification dataset instead of iris. This is a bit slimmer computationally and saves an import.

We could get rid of the old test case I'd written, but the original is a bit broader in scope as it uses BaseSearchCV for any custom CV object instead of just using GridSearchCV.

Copy link
Member

jnothman left a comment

Please add an entry to the change log at doc/whats_new/v0.22.rst. Like the other entries there, please reference this pull request with :pr: and credit yourself (and other contributors if applicable) with :user:

Copy link
Contributor

NicolasHug left a comment

a few comments but looks good

:mod:`sklearn.model_selection`
..................

- |Fix| :class:`model_selection.BaseSearchCV` now supports the

This comment has been minimized.

Copy link
@NicolasHug

NicolasHug May 28, 2019

Contributor

Please refer to GridSearchCV and RandomizedSearchCV instead.

Test implementation of BaseSearchCV has the _pairwise property
which matches the _pairwise property of its estimator.
"""
class PairwiseCV(BaseSearchCV):

This comment has been minimized.

Copy link
@NicolasHug

NicolasHug May 28, 2019

Contributor

why not using GridSearchCV instead of creating a dummy class?

def __init__(self, estimator, **kwargs):
super().__init__(estimator, **kwargs)

# first test: check BaseSearchCV children copy _pairwise

This comment has been minimized.

Copy link
@NicolasHug

NicolasHug May 28, 2019

Contributor

I'm not sure what that means.
what about:

Make sure _pairwise is delegated to the base estimator

@xun-tang

This comment has been minimized.

Copy link
Contributor

xun-tang commented Nov 2, 2019

No activity for several months, I'm picking this one up - will try to address the lingering comments.

@cmarmo

This comment has been minimized.

Copy link
Member

cmarmo commented Dec 12, 2019

@jnothman, @NicolasHug, if I understand correctly this PR has been superseded by #15524, already merged.
Should this one be closed?

@jnothman

This comment has been minimized.

Copy link
Member

jnothman commented Dec 12, 2019

Thanks @cmarmo

@jnothman jnothman closed this Dec 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
5 participants
You can’t perform that action at this time.