Add custom kernels to SpectralClustering #1791

Closed
larsmans opened this Issue Mar 18, 2013 · 13 comments

Comments

Projects
None yet
3 participants
Owner

larsmans commented Mar 18, 2013

SpectralClustering supports the RBF kernel, but it could benefit from the generic kernel support in sklearn.metric.pairwise_kernels.

KernelPCA and sklearn.kernel_approximations.Nystroem already do this and provide an example API (kernel_params for callables, custom attributes for the built-in kernels).

Contributor

rolisz commented Mar 19, 2013

Can I try to tackle this? I'd like to participate in GSOC, and this seems like an easy enough first issue to fix

Owner

larsmans commented Mar 19, 2013

Be our guest; I don't do it myself to leave it to potential GSOC participants :)

Contributor

rolisz commented Mar 19, 2013

Okay. I'll try to do fix this.

I have a question though. Both KernelPCA and sklearn.kernel_approximations.Nystroem use the constructor argument kernel for setting the kernel type, but SpectralClustering uses the affinity to choose between rbf, nearest_neighbors or precomputed. Should I change affinity to kernel or not? Or add another parameter?

Owner

larsmans commented Mar 19, 2013

I'm not an expert in kernel methods, Gaël is not available for scikit-learn stuff right now and changing a parameter name requires following a deprecation procedure, so I think the safe option is to not change the name and pass affinity to pairwise_kernels if it is not equal to nearest_neighbors.

Owner

amueller commented Mar 19, 2013

Affinity is correct in this case. SpectralClustering does not need a kernel. For example the nearest neighbor graph works fine.

Owner

larsmans commented Mar 19, 2013

The difference being that an affinity need not be symmetric, right? But then a callable needs to be handled specially (e.g. by simply disallowing it) because pairwise_kernels assumes symmetry to reduce the number of calls.

Owner

amueller commented Mar 19, 2013

The difference is that a kernel needs to be positive definite.

Owner

amueller commented Mar 19, 2013

symmetry should be assumed in both cases, I think.

Owner

larsmans commented Mar 19, 2013

A kernel needs to be pos def for an SVM, but is that a requirement for calling something a kernel? E.g. an RVM can handle "non-Mercer" kernels (not sure what properties they should still have).

You're right about the symmetry, though, this is actually enforced inside the algorithm.

Owner

amueller commented Mar 19, 2013

Mercer's condition is just positive definiteness, right?
In machine learning, I think it is understood that "kernel" means "mercer kernel".

There are also kernels in density estimation and image processing so the word is pretty overloaded.

Owner

larsmans commented Mar 19, 2013

Apparently so. Let's stick to "affinity".

The question then is what happens to kernel_params?

Contributor

rolisz commented Mar 20, 2013

@larsmans I fixed the things you mentioned about the commit, but I have a problem with some of the tests. For most of the kernels (all except poly, rbf and linear) complained about negative values in the arrays, so I moved the cluster centers away from the negative region, and I also test for equality of shapes, instead of the adjusted_rand_score. Is this okay?

And additive_chi2 still gives an error about ValueError: Array contains NaN or infinity. What should I do with it?

Owner

larsmans commented Mar 20, 2013

Let's discuss at the PR, not here.

larsmans closed this in bc67dbf Mar 30, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment