At the moment partly ignores fit_intercept if X is sparse: It sets the intercept to the mean of y but doesn't take into account X, because _center_data doesn't center sparse matrices.
This PR fits a (penalized) intercept in the sparse case by using add_dummy_feature.
Add failing test for issue #1389
Fit intercept in Ridge with dummy feature. Wide tests fail
Reduce branches, leverage `_set_intercept`
Ping @amueller and @mblondel
What do you mean by "wide X"?
@mblondel: this test passes when shape is (5, 3) but not when it's (3, 5) -- this is what I mean by wide X.
@vene When solver="auto" (the default), the solver used is not the same when X is dense and when X is sparse so you don't get the exact same results. Can you try to set the solver explicitly? (e.g. sparse_cg, which despite the name handle dense arrays too)
The unit tests are getting messy. It would be nice to break the tests into smaller units (not needed for this PR).
@mblondel: it doesn't fix it, and furthermore the results are the same, it's not that.
Does adding a column of ones change anything fundamental when the problem admits exact solutions?
@mblondel should I close this and let you take over?
I won't be able to tackle this issue this week unfortunately :-/
Feel free to finish it :-)
@vene Can I try working on this, on a new branch, and send a separate PR if you do not mind?
@vene @mblondel I do not really understand how we should center X (without breaking the sparsity or adding an extra column of ones for the intercept). Do we implement some sort of trick/hack after we get the coefficients?