head size 512 for sdpa

[syurkevi]

Description

This PR enables a head size<=512 for the SDPA microkernel. Good configurations were profiled over PVC, BMG, LNL and DG2.
Most shapes with head size 512 still give good speedups vs a primitives based implementation. Even larger head sizes may still be beneficial, more investigation to follow.

Checklist

General

• Do all unit and benchdnn tests (make test and make test_benchdnn_*) pass locally for each commit?
• Have you formatted the code using clang-format?

[syurkevi]

make test
disable test_device_cpu
disable build_cpu_runtime_omp
disable build_cpu_runtime_sycl
disable build_cpu_runtime_tbb
disable benchdnn_all
enable benchdnn_graph
enable test_device_gpu
enable arch_gpu_xe-hpc
enable arch_gpu_xe-hpg-atsm
enable arch_gpu_xe-hpg-dg2
enable arch_gpu_xe-lp
enable arch_gpu_xe-lpg
enable arch_gpu_xe-lpg+
enable arch_gpu_xe2-hpg-bmg
enable arch_gpu_xe2-lpg

[TaoLv]

@petercad @syurkevi Is it possible to further extend this to 576 to support MLA? There is the same request for flash attention.

[syurkevi]

@petercad @syurkevi Is it possible to further extend this to 576 to support MLA? There is the same request for flash attention.

576 is close enough to 512 that the current configs may be applicable as is. I've pushed a test branch internally with this change if you'd like to try it out (I have some concerns wrt/correctness). If 576 is all that's needed, I'll perform some additional testing and bump the number in this PR.

If we want even greater head sizes, at some point there will be too much register demands to be performant (or run at all depending on the hardware). Something like head_size=1024 will require more tuning+testing.

[TaoLv]

@syurkevi Good to know it. Feel free to add it in another PR. I think 576 will be enough as for now.

[atkassen]

In src/gpu/intel/ocl/micro_sdpa.cpp,

• on (new) line 79:

          if (!q_config_str.empty() && quantized) config_str = std::move(q_config_str);

• before (new) line 133:

              break;

to address some coverity issues. Thanks!

[syurkevi]

@syurkevi Good to know it. Feel free to add it in another PR. I think 576 will be enough as for now.

Bad news here after some testing.
There are no good quantized configs for head size=576. There are a few ok configs with fp16 (except DG2) that I did enable.
We'd likely need to make some updates to the microkernel generation or change the kernel to iterate over the head size w/the existing configs, both options quite involved.

[syurkevi]

make test
disable test_device_cpu
disable build_cpu_runtime_omp
disable build_cpu_runtime_sycl
disable build_cpu_runtime_tbb
disable benchdnn_all
enable benchdnn_graph
enable test_device_gpu
enable arch_gpu_xe-hpc
enable arch_gpu_xe-hpg-atsm
enable arch_gpu_xe-hpg-dg2
enable arch_gpu_xe-lp
enable arch_gpu_xe-lpg
enable arch_gpu_xe-lpg+
enable arch_gpu_xe2-hpg-bmg
enable arch_gpu_xe2-lpg