[READY] ENH - Add Gram Solver for single task Quadratic datafit #59
Conversation
|
I performed a benchmark and am convinced with the significant speedups the gram solver offers in cases However, I see that we have a huge overhead in the case of sparse datasets, which is not the case for dense data. Sparse case
Sparse case
That needs to be investigated further, I am particularly suspecting the sparse matrix multiplication and the conversion to dense as these are what differ the sparse from the dense case. |
skglm/solvers/gram_cd.py
Outdated
| grad[:] = grad_acc | ||
|
|
||
| # store p_obj | ||
| p_obj = 0.5 * w @ (scaled_gram @ w) - scaled_Xty @ w + penalty.value(w) |
There was a problem hiding this comment.
I believe the constant term is missing
There was a problem hiding this comment.
Indeed, nice catch!
|
following #59 (comment), we will tackle the big overhead that we have in the sparse case due to the computation of the gram matrix, which particularly impacts the performance of the solver, in a separate issue+PR |
Badr-MOUFAD
changed the title
This aims to add Gram solver that uses gram matrix precomputation to solve problems with quadratic datafit where the number of samples is largely greater than the number of features.
It will proceed as follows:
mainsolver