On the Snapdragon android mobile, the cblas_sbgemm function outputs all zeros.

[zlaazlaa]

The compilation options added BUILD_BFLOAT16=1. Checking /proc/cpuinfo also supports the bf16 format, but the final output matrix is all zeros.

void cpu_gemm_blas_bf16(
    const bfloat16* A, const bfloat16* B,
    size_t M, size_t K, size_t N,
    bfloat16* C,
    bool transA, bool transB)
{
	for (int i = 0; i < M * K; i++) {
		printf("A[%d] = %.6f\n", i, static_cast<float>(A[i]));
	}
	for (int i = 0; i < K * N; i++) {
		printf("B[%d] = %.6f\n", i, static_cast<float>(B[i]));
	}

    std::vector<float> C_float(M * N, 0.0f);
    
    blasint lda = transA ? M : K;
    blasint ldb = transB ? K : N;
    blasint ldc = N;
    
    cblas_sbgemm(
        CblasRowMajor,
        transA ? CblasTrans : CblasNoTrans,
        transB ? CblasTrans : CblasNoTrans,
        M,
        N,
        K,
        1.0f,
        A,
        lda,
        B,
        ldb,
        0.0f,
        C_float.data(),
        ldc
    );

	for (size_t i = 0; i < M * N; ++i) {
		printf("C_float[%zu] = %.6f\n", i, C_float[i]);
	}

    for (size_t i = 0; i < M * N; ++i) {
        C[i] = static_cast<bfloat16>(C_float[i]);
    }
}


void test_bf16() {
    const int M = 2, K = 3, N = 2;

    std::vector<bfloat16> A(M * K);
    std::vector<bfloat16> B(N * K);
    std::vector<bfloat16> C(M * N);

    float A_values[] = {1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f};
    for (int i = 0; i < M * K; ++i) A[i] = static_cast<bfloat16>(A_values[i]);

    float B_values[] = {7.0f, 8.0f, 9.0f, 10.0f, 11.0f, 12.0f};
	for (int i = 0; i < N * K; ++i) B[i] = static_cast<bfloat16>(B_values[i]);

    std::vector<float> C_float(M * N);
	cpu_gemm_blas_bf16(
		A.data(), B.data(),
		M, K, N,
		C.data(),
		false, true
	);
    
	printf("BF16 GEMM Result C (2x2):\n");
	for (int i = 0; i < M * N; ++i) {
		printf("%.2f ", static_cast<float>(C[i]));
	}
	printf("\n");
}

And the result is below:

A[0] = 1.000000
A[1] = 2.000000
A[2] = 3.000000
A[3] = 4.000000
A[4] = 5.000000
A[5] = 6.000000
B[0] = 7.000000
B[1] = 8.000000
B[2] = 9.000000
B[3] = 10.000000
B[4] = 11.000000
B[5] = 12.000000
C_float[0] = 0.000000
C_float[1] = 0.000000
C_float[2] = 0.000000
C_float[3] = 0.000000
BF16 GEMM Result C (2x2):
0.00 0.00 0.00 0.00 

I have two questions:

• In the call of bf16, what caused the result to be all zeros?
• Apart from bf16, does OpenBLAS support performing fp16 matrix multiplication calculations on ARM64 chips?

[Mousius]

Yikes, did you build with a specific TARGET ? If you set OPENBLAS_VERBOSE=2 - what core is it using?

Even if it falls back to generic kernels this seems wrong.

[Mousius]

Also, you're creating C as bfloat16:

    std::vector<bfloat16> C(M * N);

• sbgemm has bfloat16 inputs and float outputs
• bgemm has bfloat16 inputs and outputs

[martin-frbg]

I wonder if the static_cast to bfloat16 is working as expected here, we do have an sbstobf16() utility function

[zlaazlaa]

@Mousius Thanks for your reply! My target is CORTEXA57 , build command is:

execute_process(
        COMMAND make
        HOSTCC=gcc
        CC=${NDK_TOOLCHAIN_PATH}/aarch64-linux-android24-clang
        BUILD_WITHOUT_LAPACKE=0
        C_LAPACK=1
        NO_CBLAS=0
        NO_LAPACK=0
        USE_OPENMP=1
        NOFORTRAN=1
	BUILD_BFLOAT16=1
        AR=${NDK_TOOLCHAIN_PATH}/llvm-ar
        TARGET=CORTEXA57
        RANLIB=${NDK_TOOLCHAIN_PATH}/llvm-ranlib
        WORKING_DIRECTORY ${OPENBLAS_PATH}
        RESULT_VARIABLE result
        OUTPUT_FILE ${OUTPUT_FILE}
        ERROR_FILE ${ERROR_FILE}
)

And after setting OPENBLAS_VERBOSE=2, the compile failed.

clang: error: clang frontend command failed with exit code 139 (use -v to see invocation)
Android (13065274, +pgo, +bolt, +lto, +mlgo, based on r547379) clang version 20.0.0 (https://android.googlesource.com/toolchain/llvm-project b718bcaf8c198c82f3021447d943401e3ab5bd54)
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /home/mqy/Android/Sdk/ndk/29.0.13113456/toolchains/llvm/prebuilt/linux-x86_64/bin
13 warnings generated.

[zlaazlaa]

@martin-frbg Thanks! After testing, it is indeed a type conversion issue. After using the sbstobf16 function, cblas_sbgemm produced the correct value.

[zlaazlaa]

I have a small question, does OpenBLAS support matrix multiplication in fp16 format on arm64? Because I couldn't find the relevant API. My requirement is matrix multiplication in fp16 format, and my current idea is to convert fp16 to bf16 for computation, but this would result in a significant loss of precision.

[Mousius]

@zlaazlaa if you want to skip the final conversion you can still use bgemm 😸

OPENBLAS_VERBOSE is an environment variable, not a make parameter, apologies for not making that clear 😅

[zlaazlaa]

@zlaazlaa if you want to skip the final conversion you can still use bgemm 😸

OPENBLAS_VERBOSE is an environment variable, not a make parameter, apologies for not making that clear 😅

Oh! So that's how it is, I will try next. Thanks!😊

[Mousius]

I don't think there's a shgemm optimised implementation for any of the arm64 targets 🤔

[Mousius]

We also don't have an hgemm, which I presume would be better again?

[martin-frbg]

There is only the beginnings of fp16 support on RISCV64 right now, limited to SHGEMM (from #5290, so only in the develop branch so far) on cpus that support specific instruction set extensions

[zlaazlaa]

@Mousius @martin-frbg I understand, thank you all.👍