Would you share the performance data?

[oscarriddle]

I tried to realize the matrix multiplication over batched 2D input matrix A with the same 2D matrix B.

By simply expand the gridDims.z, which seems the same way like you did, I find the computation become quite slow.

[salehjg]

Hi :D

I'm assuming you have [matrix A of batch size b] and [matrix B of batch size b].
how exactly can I help?
For the default matrix sizes, this is the nvvp timeline
batch-matmul-cuda(gtx1070).nvvp.zip

Just consider that this is not the fastest possible kernel for batch tiled-matrix-multiplication.

[salehjg]

btw, it might be the tile size or the block size that hinders fast multiplication.
It should be easier to track down performance issues for a given kernel and input configuration.

[oscarriddle]

Hi, thanks for your reply.

I'm trying to design a fast batch tiled-mm kernel. In my circumstance, the batch means that multiple matrix A multiply with a single matrix B. Not multi-A multiply correspondingly with multi-B. I will take a deeper look into your idea :)

Thanks,

---

Here is my idea:

1. First, complete the tiled MM by high efficient share memory size, then start the kernel with:

gridDim(DIVUP(CC, Tile_size), DIVUP(CH*CW, Tile_size), 1), 
blockDim(Tile_size, Tile_size)

2. Above method realized a 2D MM with pretty good performance. Then I tried to do the batched As with single B, by utilizing the gridDims.z like below:

gridDim(DIVUP(CC, Tile_size), DIVUP(CH*CW, Tile_size), batch_size), 
blockDim(Tile_size, Tile_size)

The performance becomes not that good, so I wondered whether there exist some good ideas.