Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RidgeCV triggers a call to toarray on sparse matrix input #1921

Closed
ogrisel opened this issue May 3, 2013 · 4 comments
Closed

RidgeCV triggers a call to toarray on sparse matrix input #1921

ogrisel opened this issue May 3, 2013 · 4 comments
Labels

Comments

@ogrisel
Copy link
Member

ogrisel commented May 3, 2013

and thus causes a MemoryError on high dimensional data.

Details here: http://stackoverflow.com/a/16351308/163740

@mblondel
Copy link
Member

mblondel commented May 3, 2013

As I answered on SO, using gcv_mode="eigen" should allow to work around the problem.

To fix the problem in scikit-learn, we need use a proper sparse SVD when gcv_mode="svd". We can also automatically choose the "eigen" mode when gcv_mode="auto" and the data is sparse (currently "auto" uses the svd mode if n_samples > n_features, otherwise it uses the eigen mode).

BTW, I didn't write this part of the code...

@bdkearns
Copy link
Contributor

Certainly one can construct arrays that blow up any method in memory usage? So here the 'fix' was to not only make eigen the default for sparse arrays but to force all sparse inputs to eigen and actually ban them from SVD? That definitely broke other usage, see #2354

@ddofer
Copy link

ddofer commented Apr 24, 2015

Error is still occuring. (Latest version of sklearn, Windows 8.1 , MNIST data [Don't know if that counts as sparse or not])

@ogrisel
Copy link
Member Author

ogrisel commented Apr 24, 2015

@ddofer MNIST is typically loaded as a dense numpy array. This is probably a different issue. Can you please a separate issue with a minimalistic reproduction script and the full traceback you observe?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants