[WebGPU] Unify core implementations of GEMM and MatMul

[xiaofeihan1]

Description

This PR extract core implementations into gemm_utils.cc which is used to generate shader both GEMM and Matmul ops. The core implemenations included scalar and vec4 versions of GEMM and Matmul.

Motivation and Context

There are many common codes for GEMM and Matmul, so we want to extra common code to unify their implementations.

[qjia7]

LGTM.
@fs-eire @guschmue Please take a look, thanks.

[guschmue]

testing it with a bunch of models that use Gemm

[guschmue]

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 5 pipeline(s).

[guschmue]

I see some shader compile issue for couple of models in MatMul. Let me try to isolate it.

[guschmue]

In HandleMaybeBiasForMatMul:
output_element_t needs to be output_value_t ?

if (has_bias) {
shader.AdditionalImplementation() << " value = value + output_element_t(" << (is_channels_last ? "bias[colIn]" : "bias[row]") << ");\n";
}

Surprised that no unit test runs into it.

[guschmue]

With the issue above fixed locally, 90 model test now reports only numerical issues:
total: 93, setup: 92, pass: 64
pass should be around 84.

Simplest model with numerical issues:
https://huggingface.co/onnx-community/mobilenetv4_conv_small.e2400_r224_in1k

[xiaofeihan1]

Surprised that no unit test runs into it.

Thanks for the callout!!! I will add some test cases to cover the scenario. Before there is only small matrix(M,N,K<8) with bias test case.

[xiaofeihan1]

With the issue above fixed locally, 90 model test now reports only numerical issues: total: 93, setup: 92, pass: 64 pass should be around 84.

Simplest model with numerical issues: https://huggingface.co/onnx-community/mobilenetv4_conv_small.e2400_r224_in1k

Thanks for helping with the testing! I'll look into why there's a problem.

[fs-eire]

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 5 pipeline(s).

[xiaofeihan1]

With the issue above fixed locally, 90 model test now reports only numerical issues: total: 93, setup: 92, pass: 64 pass should be around 84.

Simplest model with numerical issues: https://huggingface.co/onnx-community/mobilenetv4_conv_small.e2400_r224_in1k

Thanks Guenther! I have fixed the numerical issue and add more test to cover this scenario. Could you help me to run the 90 model test again with the latest commit?

[fs-eire]

seems like build break with the following error:

/onnxruntime_src/onnxruntime/test/providers/cpu/math/gemm_test.cc:1060:15: error: unused variable ‘bias_value’ [-Werror=unused-variable]
 1060 |         float bias_value = 0.0f;
      |               ^~~~~~~~~~

[xiaofeihan1]

seems like build break with the following error:

/onnxruntime_src/onnxruntime/test/providers/cpu/math/gemm_test.cc:1060:15: error: unused variable ‘bias_value’ [-Werror=unused-variable]
 1060 |         float bias_value = 0.0f;
      |               ^~~~~~~~~~

Thanks. Done

[fs-eire]

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 5 pipeline(s).

[guschmue]

let me kick the 90 model test

[guschmue]

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 5 pipeline(s).

[guschmue]

90 model test looks good now

[xiaofeihan1]

90 model test looks good now
Cool!!Thanks Guenther for testing!
@fs-eire Let’s move forward with merging this PR. Any comment is welcome.