about zenMatMulSplit split strategy

[MichoChan]

" if m/6 < n/16, there is no benifit in splitting m, as it will
//make more skinny matix sizes."

i find that throughput is higher when no split in case (m/6 < n/16), but latency is higher than split, could you give suggestions about how to balance the throughput with latency ? is there some better split strategy ?

[cramasam]

We don't do split on m dimension in ZenDNN based on following conditions,

1. (m/6 < n/16) - Splitting here will make the matrix more skinny. BLIS internally handles in optimal way
2. (n,n,k >= 1024) - This makes matrix more suitable for BLIS to take decision of split/no-split internally
3. (m,n >= 4096) - Same for this condition like '2'
4. (m <= (thread_qty6) - m is not large enough to accommodate total threads6 size

For latency we have been observed that BLIS takes cares of it in the optimal way. We just need to bypass the call to BLIS sgemm.

If you can share the sizes of matrices that you are working on, It would help us to give you better insights.