Speed up CuMatrix<Real>::Transpose() and transposed copy from matrix

[kangshiyin]

A new matrix transpose kernel. 2~3x faster. Benchmark results are in the commit log.

[danpovey]

Thanks!
Please show us the speed difference here in the conversation so we don't have to look at the log.
I notice you have a template that can take 16 or 32 but you're only using 32. I recommend simplifying the code if you don't have immediate plans to use that functionality.
Dan

[kangshiyin]

OK, will change. Here's the timing.

Original:
LOG (TestCuMatrixTransposeCross():cu-matrix-speed-test.cc:91) For CuMatrix::TransposeCross, for dim = 1024, speed was 4.26727 gigaflops.
LOG (TestCuMatrixTransposeS():cu-matrix-speed-test.cc:72) For CuMatrix::TransposeS, for dim = 1024, speed was 5.97203 gigaflops.
LOG (TestCuMatrixTransposeNS():cu-matrix-speed-test.cc:56) For CuMatrix::TransposeNS, for dim = 1024, speed was 3.0816 gigaflops.
LOG (TestCuMatrixTransposeCross():cu-matrix-speed-test.cc:91) For CuMatrix::TransposeCross, for dim = 1024, speed was 3.95059 gigaflops.
LOG (TestCuMatrixTransposeS():cu-matrix-speed-test.cc:72) For CuMatrix::TransposeS, for dim = 1024, speed was 4.36189 gigaflops.
LOG (TestCuMatrixTransposeNS():cu-matrix-speed-test.cc:56) For CuMatrix::TransposeNS, for dim = 1024, speed was 2.39275 gigaflops.
c765ba6

New:
LOG (TestCuMatrixTransposeCross():cu-matrix-speed-test.cc:91) For CuMatrix::TransposeCross, for dim = 1024, speed was 14.0498 gigaflops.
LOG (TestCuMatrixTransposeS():cu-matrix-speed-test.cc:72) For CuMatrix::TransposeS, for dim = 1024, speed was 16.845 gigaflops.
LOG (TestCuMatrixTransposeNS():cu-matrix-speed-test.cc:56) For CuMatrix::TransposeNS, for dim = 1024, speed was 14.2464 gigaflops.
LOG (TestCuMatrixTransposeCross():cu-matrix-speed-test.cc:91) For CuMatrix::TransposeCross, for dim = 1024, speed was 10.4523 gigaflops.
LOG (TestCuMatrixTransposeS():cu-matrix-speed-test.cc:72) For CuMatrix::TransposeS, for dim = 1024, speed was 9.65529 gigaflops.
LOG (TestCuMatrixTransposeNS():cu-matrix-speed-test.cc:56) For CuMatrix::TransposeNS, for dim = 1024, speed was 8.52148 gigaflops.