Question about the cublaslt_gemm.cu

[south-ocean]

In the cublas_gemm function of the transformer_engine/common/gemm/cublaslt_gemm.cu file, I see that CUBLALLT is called, but I have a question, that is, if it is pure GEMM, that is, there is no bias or activation behind GEMM, then does it make sense to use CUBLALT in the BLAS function? In my opinion, the upper limit of cublaslt performance is cublas under pure gemm call, so why not take cublas in non-fusion mode and cublaslt in fusion mode?

[yaox12]

1. Only cuBLASLt supports FP8 GEMM.
2. For pure GEMMs, cuBLAS calls cuBLASLt internally, so I think there's no difference in terms of performance, while cuBLASLt offers more controls.

[south-ocean]

Thanks for the answer, but I still have a little doubt about the second point, you mentioned For pure GEMMs, cuBLAS calls cuBLASLt internally, so I think there's no difference in terms of performance, while cuBLASLt offers more controls.
1、 In my opinion, cublas calls blaslt for programming convenience, right? and Doesn't CUBLAS call blaslt cause performance degradation?
2、As mentioned above, the upper limit of performance for pure GEM CUBLalt should be BLAS, and now all the operators in CUBLAS follow the implementation of CUBLASLT?

[yaox12]

Why does calling cuBLASLt cause performance degradation? For GEMMs with the same input data types and shapes, cuBLAS and cuBLASLt should select the same kernel and have the same performance.

now all the operators in CUBLAS follow the implementation of CUBLASLT?

cuBLAS provides a broader range of linear algebra functions other than GEMMs that are not covered in cuBLASLt.