CUDA sparse-matrix class

[danpovey]

Something needs to be coded for sparse-matrix support that's going to be quite important for the new 'chain' (post-CTC) stuff I am working on, and maybe some nearer-term things too (relating to multi-tree systems- we should be able to get a nice improvement).
We need to be able to efficiently do matrix multiplication by a sparse matrix. The current CuSparseMatrix class does not have a data format that is conducive to this.
in cu-sparse-matrix.h, I'd like a new class added, that stores its data per-row.

The class CuRowSparseMatrix will have members:

int32 num_rows_;
int32 elements_per_row_;
CuArray < std::pair<int32, Real> > data_; // pairs (column-index, value).
//dim = num_rows_ * elements_per_row_.

where elements_per_row_ will be like a stride; it will be the maximum number of nonzero elements per row; you'll pad with (-1, 0.0) in case some rows have fewer than the maximum number of elements.
You can pass the pointer to the data across the "C" interface as void*. It's messy but it avoids having to create a "C" equivalent to the C++ pair.

The key function that I want written (aside from the one to create the CuRowSparseMatrix from a SparseMatrix is the following function:

// Do *this = alpha * a * b + beta * *this.
// Only supports the case where trans_b = kTrans; will crash if trans_b = kNoTrans.
void CuMatrixBase::AddMatSmat(Real alpha, CuMatrixBase &a, MatrixTransposeType trans_a,
CuRowSparseMatrix &b, MatrixTransposeType trans_b, Real beta);

The kernel will do the summation over the elements_per_row_ elements using a simple for loop. You can assume that elements_per_row_ is fairly small, otherwise we wouldn't be using this.

Obviously there needs to be test code for this.

[danpovey]

@naxingyu, do you have time for this? I know you've done a lot lately..

[naxingyu]

I'll do this next week (this week on phone-topology).

[danpovey]

OK thanks-- check in with us when you've done the phone-topology thing
though, because @hainan-xv was considering doing it.

On Wed, Oct 28, 2015 at 10:28 PM, Xingyu Na notifications@github.com
wrote:

I'll do this next week (this week on phone-topology).

—
Reply to this email directly or view it on GitHub
#298 (comment).

[naxingyu]

I'll create a interface for CuRowSparseMatrix similar to CuSparseMatrix.
Some basics are

• return numbers
• copy and swap functions
• data accessors and randomizer
• operators
• read and write
• a constructor from a SparseMatrix

Besides those, should I implement these?

• sum
• frobenius norm

And according to your design, I will add AddMatSmat as a member function
of CuMatrixBase, is that right?

Xingyu

On 10/29/2015 10:31 AM, Daniel Povey wrote:

OK thanks-- check in with us when you've done the phone-topology thing
though, because @hainan-xv was considering doing it.

On Wed, Oct 28, 2015 at 10:28 PM, Xingyu Na notifications@github.com
wrote:

I'll do this next week (this week on phone-topology).

—
Reply to this email directly or view it on GitHub
#298 (comment).

—
Reply to this email directly or view it on GitHub
#298 (comment).

[danpovey]

I'll create a interface for CuRowSparseMatrix similar to CuSparseMatrix.
Some basics are

• return numbers
• copy and swap functions
• data accessors and randomizer
• operators
• read and write
• a constructor from a SparseMatrix

sure.

Besides those, should I implement these?

• sum
• frobenius norm

only if you have time-- they might be useful for some diagnostics. but we
can always copy to CPU for this.

And according to your design, I will add AddMatSmat as a member function
of CuMatrixBase, is that right?

yes.
Dan

Xingyu

On 10/29/2015 10:31 AM, Daniel Povey wrote:

OK thanks-- check in with us when you've done the phone-topology thing
though, because @hainan-xv was considering doing it.

On Wed, Oct 28, 2015 at 10:28 PM, Xingyu Na notifications@github.com
wrote:

I'll do this next week (this week on phone-topology).

—
Reply to this email directly or view it on GitHub
<#298 (comment)
.

—
Reply to this email directly or view it on GitHub
#298 (comment).

—
Reply to this email directly or view it on GitHub
#298 (comment).

[naxingyu]

It seems I should create a RowSparseMatrix class too? To handle the
non-cuda situation?

X.

On 11/04/2015 11:59 AM, Daniel Povey wrote:

I'll create a interface for CuRowSparseMatrix similar to CuSparseMatrix.
Some basics are

• return numbers
• copy and swap functions
• data accessors and randomizer
• operators
• read and write
• a constructor from a SparseMatrix

sure.

Besides those, should I implement these?

• sum
• frobenius norm

only if you have time-- they might be useful for some diagnostics. but we
can always copy to CPU for this.

And according to your design, I will add AddMatSmat as a member function
of CuMatrixBase, is that right?

yes.
Dan

Xingyu

On 10/29/2015 10:31 AM, Daniel Povey wrote:

OK thanks-- check in with us when you've done the phone-topology thing
though, because @hainan-xv was considering doing it.

On Wed, Oct 28, 2015 at 10:28 PM, Xingyu Na notifications@github.com
wrote:

I'll do this next week (this week on phone-topology).

—
Reply to this email directly or view it on GitHub

<#298 (comment)
.

—
Reply to this email directly or view it on GitHub

#298 (comment).

—
Reply to this email directly or view it on GitHub
#298 (comment).

—
Reply to this email directly or view it on GitHub
#298 (comment).

[danpovey]

Actually no, because the regular SparseMatrix class already stores its data
by row.
Dan

On Thu, Nov 5, 2015 at 4:09 AM, Xingyu Na notifications@github.com wrote:

It seems I should create a RowSparseMatrix class too? To handle the
non-cuda situation?

X.

On 11/04/2015 11:59 AM, Daniel Povey wrote:

I'll create a interface for CuRowSparseMatrix similar to
CuSparseMatrix.
Some basics are

• return numbers
• copy and swap functions
• data accessors and randomizer
• operators
• read and write
• a constructor from a SparseMatrix

sure.

Besides those, should I implement these?

• sum
• frobenius norm

only if you have time-- they might be useful for some diagnostics. but we
can always copy to CPU for this.

And according to your design, I will add AddMatSmat as a member
function
of CuMatrixBase, is that right?

yes.
Dan

Xingyu

On 10/29/2015 10:31 AM, Daniel Povey wrote:

OK thanks-- check in with us when you've done the phone-topology
thing
though, because @hainan-xv was considering doing it.

On Wed, Oct 28, 2015 at 10:28 PM, Xingyu Na <
notifications@github.com>
wrote:

I'll do this next week (this week on phone-topology).

—
Reply to this email directly or view it on GitHub

<#298 (comment)
.

—
Reply to this email directly or view it on GitHub

#298 (comment).

—
Reply to this email directly or view it on GitHub
<#298 (comment)
.

—
Reply to this email directly or view it on GitHub
#298 (comment).

—
Reply to this email directly or view it on GitHub
#298 (comment).