Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "Randomized SVD" solver option to KernelPCA for faster partial decompositions, like in PCA #12068

Open
smarie opened this issue Sep 13, 2018 · 7 comments · May be fixed by #12069
Open

Add "Randomized SVD" solver option to KernelPCA for faster partial decompositions, like in PCA #12068

smarie opened this issue Sep 13, 2018 · 7 comments · May be fixed by #12069

Comments

@smarie
Copy link
Contributor

@smarie smarie commented Sep 13, 2018

PCA currently offers an svd_solver parameter allowing users to get faster partial decomposition using two methods: 'arpack' and 'randomized' svd.

KernelPCA offers an eigen_solver parameter allowing users to get faster partial decomposition using 'arpack', but 'randomized' svd is not supported. Would it seem reasonable to add it in the list of possible solvers ?

(Thanks grilling for the suggestion and support for implementation !)

smarie added a commit to smarie/scikit-learn that referenced this issue Sep 13, 2018
…s proposed in scikit-learn#12068.

It works exactly the same than for PCA: simply set eigen_solver to 'randomized' to use that method.
@jnothman

This comment has been minimized.

Copy link
Member

@jnothman jnothman commented Sep 15, 2018

@jnothman

This comment has been minimized.

Copy link
Member

@jnothman jnothman commented Sep 15, 2018

@lobpcg

This comment has been minimized.

Copy link

@lobpcg lobpcg commented Sep 24, 2018

@smarie

This comment has been minimized.

Copy link
Contributor Author

@smarie smarie commented Sep 25, 2018

Cool suggestion! I see that 'amg' is also mentioned in the doc, is there any reason why we should not add it in kPCA too ?

@lobpcg

This comment has been minimized.

Copy link

@lobpcg lobpcg commented Sep 25, 2018

'amg' is an option for 'lobpcg' that may make LOBPCG work faster, but the effect is highly problem dependent. It is unclear to me if the 'amg' option would work well for kPCA. Having LOBPCG already added to kPCA, it would be easy to test 'amg' and check it is worth adding 'amg' to kPCA.

@smarie

This comment has been minimized.

Copy link
Contributor Author

@smarie smarie commented Sep 26, 2018

ok then. I suggest to wait for the current pull request to be merged first, since it already depends on two others that will be relevant in the case of 'lobpcg for kPCA' (#12145 concerns eigenvalues validation, and #12143 is to fix kPCA behaviour in case of perfect zero-eigenvalues). It will be very easy to add the new method to the benchmark provided in #12069 so that you have a nice plot of execution times.

@lobpcg

This comment has been minimized.

Copy link

@lobpcg lobpcg commented Oct 4, 2018

I added LOBPCG as an SVD solver to PCA in #12079 and to Truncated PCA in #12080, see pull request #12291 . So adding LOBPCG to kPCA should now probably be especially simple.

EDIT: #12291 closed, instead see #12319

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.