Sparse Matrix(compressed) * Dense Matrix product

[albertzaharovits]

Note: needs review!!!

[karlrupp]

Thanks, Albert! The overall structure looks really good :-) Particularly it's great that you have an implementation for all three backends now. As for the kernels, I'll comment the commit inline. You can then either commit on top of the existing commit, or force-push an amended commit.

[karlrupp]

I think you should limit the maximum size of shared memory used. If I ramp up the number of columns in the dense matrix to e.g. 3000, the GPU is easily out of shared memory.

[karlrupp]

Nice! You can optimize it a little further by interchanging the loops for 'col' and 'k', because then 'x' needs to be loaded only once. Right now you might get loaded multiple times.

[karlrupp]

Cool, almost done. Just a few minor optimizations left. :-)

[ptillet]

Hi Albert! Great job, really !

I would have some suggestions for potential performance improvement, but I think it would be wiser to first have benchmarks to measure whether performance improvements are necessary.
However, how could we benchmark it? Should we compare against other libraries or compute FLOPs, keeping in mind that the number of floating point operation is data-dependent. I have not enough experience to give an order of magnitude of the sparsity coefficient threshold at which the operation is no longer bandwidth-limited... In all cases, I think this patch would be even better if you could provide a simple benchmark in addition to the tests ;)

[karlrupp]

@PhilippeTillet For a general sparse-dense multiplication this will always be memory bandwidth limited. To make it more compute-limited we would need to introduce some block-CSR format or similar.

[karlrupp]

Thanks, Albert!