[WIP] Ridge fit_intercept with sparse X (issue #1389)

[vene]

At the moment partly ignores fit_intercept if X is sparse: It sets the intercept to the mean of y but doesn't take into account X, because _center_data doesn't center sparse matrices.

This PR fits a (penalized) intercept in the sparse case by using add_dummy_feature.

TODO:

• Default values as suggested by @mblondel
• Intercept weight
• Figure out why this doesn't work in the wide X case: either what I'm doing is wrong, or the test isn't appropriate?

See #1389

[vene]

Ping @amueller and @mblondel

[mblondel]

What do you mean by "wide X"?

[mblondel]

@vene When solver="auto" (the default), the solver used is not the same when X is dense and when X is sparse so you don't get the exact same results. Can you try to set the solver explicitly? (e.g. sparse_cg, which despite the name handle dense arrays too)

The unit tests are getting messy. It would be nice to break the tests into smaller units (not needed for this PR).

[vene]

@mblondel: it doesn't fix it, and furthermore the results are the same, it's not that.

Does adding a column of ones change anything fundamental when the problem admits exact solutions?

[vene]

@mblondel should I close this and let you take over? 🙇

[mblondel]

I won't be able to tackle this issue this week unfortunately :-/
Feel free to finish it :-)

[MechCoder]

@vene Can I try working on this, on a new branch, and send a separate PR if you do not mind?

[vene]

Of course, please do!

On Mon, Aug 11, 2014 at 4:02 PM, Manoj Kumar notifications@github.com
wrote:

@vene https://github.com/vene Can I try working on this, on a new
branch, and send a separate PR if you do not mind?

—
Reply to this email directly or view it on GitHub
#1560 (comment)
.

[MechCoder]

@vene @mblondel I do not really understand how we should center X (without breaking the sparsity or adding an extra column of ones for the intercept). Do we implement some sort of trick/hack after we get the coefficients?

[amueller]

fixed by #13995 from what I can tell