Skip to content

graph: backend: dnnl: fix decompose kernel select index check #3425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 19, 2025

Conversation

xiang1guo
Copy link
Contributor

Fix MFDNN-13765.

Add check logic to fall back unsupported sdpa case to large_partition kernel.

ONEDNN_VERBOSE=1,filter=graph KMP_AFFINITY=compact,1,0,granularity=fine OMP_NUM_THREADS=4 OMP_PLACES=threads OMP_PROC_BIND=close numactl --membind=
0 --physcpubind=0-3 ./tests/benchdnn/benchdnn --graph --dt=7:f32+8:f32+9:f32+10:f32  --case=/home/sdp/xiangguo/onednn/oneDNN/mha-pt-starcoder-inf-bf16-bs1.json.json
onednn_verbose,v1,info,oneDNN v3.9.0 (commit 8b485fb2210a8f92b7ea2175c0e4e5692b6fc79e)
onednn_verbose,v1,info,cpu,runtime:OpenMP,nthr:4
onednn_verbose,v1,info,cpu,isa:Intel AVX-512 with float16, Intel DL Boost and bfloat16 support and Intel AMX with bfloat16 and 8-bit integer support
onednn_verbose,v1,info,gpu,runtime:none
onednn_verbose,v1,info,graph,backend,0:dnnl_backend
onednn_verbose,v1,primitive,info,template:operation,engine,primitive,implementation,prop_kind,memory_descriptors,attributes,auxiliary,problem_desc,exec_time
onednn_verbose,v1,graph,info,template:operation,engine,partition_id,partition_kind,op_names,data_formats,logical_tensors,fpmath_mode,implementation,backend,exec_time
onednn_verbose,v1,common,info,filter format is enabled, hit components: graph
onednn_verbose,v1,graph,exec,cpu,100002,sdp,aten::matmul;aten::mul;aten::where;aten::softmax;aten::matmul,,in0_bf16:5:strided:variable:1x117x48x128:718848s6144s128s1 in1_bf16:6:strided:variable:1x1x128x117:14976s14976s117s1 in2_bf16:2:strided:undef:0:2 in3_boolean:1:strided:variable:1x117x1x117:13689s117s117s1 in4_f32:9:strided:undef:0:2 in5_bf16:19:strided:variable:1x1x117x128:14976s14976s128s1 out0_bf16:20:strided:variable:1x117x48x128:718848s6144s128s1,fpm:strict,larger_partition_kernel_t,dnnl_backend,47.416
[8743][0:1:20:39] exp_f32:  0.00468446 exp:  0.00469971 got:  0.00466919 diff:3.05176e-05 rdiff:0.00649351
[9309][0:1:24:93] exp_f32: 2.80984e-05 exp: 2.81334e-05 got:  2.7895e-05 diff:2.38419e-07 rdiff:0.00847458
[9328][0:1:24:112] exp_f32: 1.99666e-05 exp:  1.9908e-05 got: 2.00272e-05 diff:1.19209e-07 rdiff:0.00598802
[41347][0:6:35:3] exp_f32:   -0.908203 exp:   -0.910156 got:    -0.90625 diff:0.00390625 rdiff:0.00429185
[154787][0:25:9:35] exp_f32: 1.11755e-05 exp: 1.11461e-05 got: 1.12057e-05 diff:5.96046e-08 rdiff:0.00534759
[156925][0:25:25:125] exp_f32: 1.78026e-05 exp: 1.77622e-05 got: 1.78814e-05 diff:1.19209e-07 rdiff:0.00671141
[175248][0:28:25:16] exp_f32: 1.11148e-05 exp: 1.10865e-05 got: 1.11461e-05 diff:5.96046e-08 rdiff:0.00537634
[186574][0:30:17:78] exp_f32:   -0.873047 exp:   -0.871094 got:      -0.875 diff:0.00390625 rdiff:0.00448431
[187039][0:30:21:31] exp_f32:   -0.118896 exp:   -0.119141 got:   -0.118652 diff:0.000488281 rdiff:0.00409836
[220921][0:35:45:121] exp_f32:    -1.77734 exp:    -1.78125 got:    -1.77344 diff:0.0078125 rdiff:0.00438596
[COMPARE_STATS]: trh=0 err_max_diff:0.0078125 err_max_rdiff:0.00847458 all_max_diff:0.0078125 all_max_rdiff:0.294304
[COMPARE_STATS] Norm check is allowed; error_to_total_ratio: 24/718848; allowed_ratio: 702/718848;
0:PASSED (316 ms) __REPRO: --graph --dt=7:f32+8:f32+9:f32+10:f32 --case=/home/sdp/xiangguo/onednn/oneDNN/mha-pt-starcoder-inf-bf16-bs1.json.json
tests:1 passed:1 skipped:0 mistrusted:0 unimplemented:0 invalid_arguments:0 failed:0 listed:0
total: 0.32s; create_pd: 0.00s (0%); create_prim: 0.00s (0%); fill: 0.00s (0%); execute: 0.00s (0%); compute_ref: 0.00s (0%); compare: 0.00s (0%);

@xiang1guo xiang1guo requested a review from a team as a code owner June 16, 2025 08:12
@github-actions github-actions bot added the component:graph-api Codeowner: @oneapi-src/onednn-graph label Jun 16, 2025
@xiang1guo xiang1guo self-assigned this Jun 16, 2025
@ShanSimu
Copy link

ShanSimu commented Jun 16, 2025

I have validated that it works for the first case in MFDNN-13765

@xiang1guo
Copy link
Contributor Author

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@xiang1guo xiang1guo force-pushed the xiang/main/fix-sdpa-decompose branch from 0257927 to 53ae284 Compare June 17, 2025 02:36
@xiang1guo xiang1guo force-pushed the xiang/main/fix-sdpa-decompose branch from 53ae284 to ad2f9bc Compare June 18, 2025 00:51
@xiang1guo
Copy link
Contributor Author

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@xiang1guo xiang1guo force-pushed the xiang/main/fix-sdpa-decompose branch from ad2f9bc to 0dbdf9f Compare June 18, 2025 02:07
@xiang1guo
Copy link
Contributor Author

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@TaoLv TaoLv merged commit 5379207 into main Jun 19, 2025
13 of 14 checks passed
@TaoLv TaoLv deleted the xiang/main/fix-sdpa-decompose branch June 19, 2025 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:graph-api Codeowner: @oneapi-src/onednn-graph
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants