No copy gems

[jagrit06]

Proposed changes

• Collapse batch dimensions in gemm
• Update gemm to only make copies when neither of the last 2 axes are contiguous

Checklist

Put an x in the boxes that apply.

• I have read the CONTRIBUTING document
• I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
• I have added tests that prove my fix is effective or that my feature works
• I have updated the necessary documentation (if needed)

[jagrit06]

Some numbers (Done with a simple multi-query attention)"

Before:

 B,   D, nkh, factor,  qsl,  ksl,   dtype, time(s)
 4,  64,  16,      4,    1, 1024, float32, 0.11741
 4,  64,  16,      4,   32, 1024, float32, 0.15783
 4,  64,  16,      4, 1024, 1024, float32, 0.87965
 4,  64,  16,      4, 1024,    1, float32, 0.16385
 4, 128,   8,      4,    1, 1024, float32, 0.17190
 4, 128,   8,      4,   32, 1024, float32, 0.15203
 4, 128,   8,      4, 1024, 1024, float32, 0.69943
 4, 128,   8,      4, 1024,    1, float32, 0.14400

After:

 B,   D, nkh, factor,  qsl,  ksl,   dtype, time(s)
 4,  64,  16,      4,    1, 1024, float32, 0.03891
 4,  64,  16,      4,   32, 1024, float32, 0.07162
 4,  64,  16,      4, 1024, 1024, float32, 0.75681
 4,  64,  16,      4, 1024,    1, float32, 0.13285
 4, 128,   8,      4,    1, 1024, float32, 0.13510
 4, 128,   8,      4,   32, 1024, float32, 0.16739
 4, 128,   8,      4, 1024, 1024, float32, 0.58777
 4, 128,   8,      4, 1024,    1, float32, 0.12501

[awni]

LGTM!! 🚀