[MRG] Fix n_components kwarg missing in SpectralClustering. See #13698

[fdas3213]

Reference Issues/PRs

Fixes #13698

What does this implement/fix? Explain your changes.

The SpectralClustering class does not have an instance variable n_components; that's why n_components cannot be accessed by an instance of SpectralClustering class. My temporary fix is to add n_components as an instance variable of SpectralClustering class, and add it as an argument of spectral_clustering in fit.
If this is a reasonable fix of the problem, I will then add a test function def test_n_components in test_spectral.py. Otherwise please let me know what is the best way to fix this.

Any other comments?

[fdas3213]

Assertion error happens at the last line assert param_value == init_param.default, init_param.name in check_parameters_default_constructible in sklearn/utils/estimator_checks.py because param_value = n_clusters if n_components is None, but init_param.default = None, and because n_clusters != None, this error occurs.

I tried to fix this by setting self.n_components = n_components in the constructor of SpectralClustering class and then change the value of self.n_components to self.n_cluster in fit function, however by doing this, I got another error Estimator SpectralClustering should not change or mutate the parameter n_components from None to 8 during fit, which basically says that I cannot change the value of instance variable outside the constructor.

These two errors might be the reason why n_components is not set an instance variable for SpectralClustering class.

So does anyone have any idea on how to handle this issue?

[thomasjpfan]

Set self.n_components = n_components and pass self.n_components directly to spectral_clustering without modifying any of the parameters in __init__. spectral_clustering already handles the case when n_components is defined:

scikit-learn/sklearn/cluster/spectral.py

Line 256 in 8d3b4ff

n_components = n_clusters if n_components is None else n_components

[fdas3213]

Nice idea! thanks for help

[fdas3213]

@thomasjpfan When I run pytest locally, there is no DocTestFailure; however when running checks after submitting new commit, I got 2 DocTestFailure. Do you know why this happens and how should I fix this problem? Thank you

[thomasjpfan]

Merge with master

[fdas3213]

Merge with master

I think the branch is already up to date with the master. Could you please be more specific on how should I do this? Thanks

[thomasjpfan]

git checkout LOCAL_BRANCH_NAME
git remote add upstream https://github.com/scikit-learn/scikit-learn.git
git fetch upstream
git merge upstream/master

We updated how we build docs in circleci, you need to merge with master to build the docs without error.

[fdas3213]

git checkout LOCAL_BRANCH_NAME
git remote add upstream https://github.com/scikit-learn/scikit-learn.git
git fetch upstream
git merge upstream/master

We updated how we build docs in circleci, you need to merge with master to build the docs without error.

thanks so much for help

@thomasjpfan
Seems that DocTestFailure still exists after merging with master.
I fixed this error locally by changing
array([1, 1, 1, 0, 0, 0])
to
array([1, 1, 1, 0, 0, 0], dtype=int64) and everything looks good locally when tested with pytest.
But even after merging with master, this DocTestFailure still exists. Any help is appreciated!

[fdas3213]

@thomasjpfan There is still that doc error after merging with master. Do you have any hint about this? Or is there anything that I did not do right? thank you

[thomasjpfan]

Revert this change

[fdas3213]

@thomasjpfan I think this PR is ready to be reviewed. Thank you

[jnothman]

Please add a brief test to check that n_components affects the result, and that it is equivalent to n_clusters by default.

[jnothman]

Suggested change

	n_components : integer, optional, default is n_clusters
	n_components : integer, optional, default=n_clusters

[fdas3213]

Sure. Should I add the test as def test_n_components in test_spectral.py?

[jnothman]

Yes, please!

[fdas3213]

To test that n_components is equivalent to n_clusters by default, I need to initialize the value of n_components in __init__ as self.n_components = n_clusters if n_components is None else n_components.

However, this would lead to Assertion error that happens at the last line assert param_value == init_param.default, init_param.name in check_parameters_default_constructible in sklearn/utils/estimator_checks.py because param_value = n_clusters if n_components is None, but init_param.default = None, and because n_clusters != None, this error occurs.

As thomasjpfan suggested, I "Set self.n_components = n_components and pass self.n_components directly to spectral_clustering without modifying any of the parameters in __init__. " But in this way, I cannot test n_components is equivalent to n_clusters by default, since self.n_components = None and self.n_clusters = n_clusters.

Any suggestion on this? Thank you

[jnothman]

I just mean that you can test that the result of the model is the same as if you set n_components=n_clusters. At this point you should only be modifying test files, no?

[fdas3213]

I just mean that you can test that the result of the model is the same as if you set n_components=n_clusters. At this point you should only be modifying test files, no?

Got it. Thanks for the clarification. Yes, only test files will be modified.

[jnothman]

Good test, thanks!!

[fdas3213]

Good test, thanks!!

No problem, glad it gets approved. There were still 2 checks failing, do I have to do something?

[jnothman]

Those failures are happening on all pull requests. Will be fixed by us

[NicolasHug]

LGTM,

Please add an entry to the change log at doc/whats_new/v0.22.rst. Like the other entries there, please reference this pull request with :pr: and credit yourself (and other contributors if applicable) with :user:.

[jnothman]

I'd call this an enhancement. Fix indicates that someone's code was previously not functioning as documented, ordinarily. Rather, you provide a way to access existing capabilities

[fdas3213]

Thanks for letting me know. Just made the change