Skip to content

Conversation

Jiexin-Zheng
Copy link
Contributor

Description

Create matmul src and weight mds with any layout only when they are constant

Fix MFDNN-13714

Performance improvements

benchdnn graph

HW mha cases speedup
PVC 1.21x
BMG 1.17x
Arc-A770 1.30x
SPR 1.01x

Pytorch GPU model

model dtype HW inductor speedup eager speedup
GPT2ForSequenceClassification fp32 PVC 1.06203 1.05513
GPT2ForSequenceClassification fp16 PVC 1.39628 1.35299
GPT2ForSequenceClassification bf16 PVC 1.40592 1.35395
PLBartForConditionalGeneration fp32 PVC 1.07212 0.99772
PLBartForConditionalGeneration fp16 PVC 1.03491 1.00943
PLBartForConditionalGeneration bf16 PVC 1.02930 1.00978
MegatronBertForCausalLM fp32 PVC 1.01970 0.99272
MegatronBertForCausalLM fp16 PVC 1.03874 0.99610
MegatronBertForCausalLM bf16 PVC 0.99199 1.00457
hf_GPT2 fp32 PVC 1.03786 1.03452
hf_GPT2 fp16 PVC 1.44831 1.40425
hf_GPT2 bf16 PVC 1.44748 1.40106
llama_v2_7b_16h fp32 PVC 1.00268 1.00336
llama_v2_7b_16h fp16 PVC 1.01445 1.00231
llama_v2_7b_16h bf16 PVC 1.06454 1.01447

@Jiexin-Zheng Jiexin-Zheng self-assigned this Jun 6, 2025
@Jiexin-Zheng Jiexin-Zheng requested a review from a team as a code owner June 6, 2025 08:37
@Jiexin-Zheng Jiexin-Zheng added the component:graph-api Codeowner: @oneapi-src/onednn-graph label Jun 6, 2025
@TaoLv TaoLv changed the title make matmul use any layout format only for constant cases graph: backend: dnnl: make matmul use any layout format only for constant cases Jun 6, 2025
@Jiexin-Zheng Jiexin-Zheng force-pushed the jiexin-zheng/main/matmul_perf_final branch 2 times, most recently from a11dd13 to 0fc7ee3 Compare June 11, 2025 03:46
@Jiexin-Zheng
Copy link
Contributor Author

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@rongzha1 rongzha1 force-pushed the jiexin-zheng/main/matmul_perf_final branch from 0fc7ee3 to 3b59626 Compare June 20, 2025 02:18
@github-actions github-actions bot added the component:tests Codeowner: @oneapi-src/onednn-arch label Jun 20, 2025
@Jiexin-Zheng Jiexin-Zheng force-pushed the jiexin-zheng/main/matmul_perf_final branch 2 times, most recently from e27f315 to e67cff5 Compare June 20, 2025 02:44
@Jiexin-Zheng
Copy link
Contributor Author

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@Jiexin-Zheng Jiexin-Zheng force-pushed the jiexin-zheng/main/matmul_perf_final branch from e67cff5 to 812cc2f Compare June 23, 2025 02:13
@Jiexin-Zheng
Copy link
Contributor Author

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@Jiexin-Zheng Jiexin-Zheng force-pushed the jiexin-zheng/main/matmul_perf_final branch from 812cc2f to c22302d Compare June 26, 2025 08:50
@Jiexin-Zheng
Copy link
Contributor Author

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@TaoLv TaoLv merged commit d863a86 into main Jun 29, 2025
23 of 24 checks passed
@TaoLv TaoLv deleted the jiexin-zheng/main/matmul_perf_final branch June 29, 2025 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:graph-api Codeowner: @oneapi-src/onednn-graph component:tests Codeowner: @oneapi-src/onednn-arch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants