[Feature]: Move query quantization to attention layer for all backends supporting query quantization.

[jmkuebler]

🚀 The feature, motivation and pitch

#24914 moved the query quantization out of the FlashAttn backend into attention/layer such that it can be fused by torch compile.
This should be done for all backends.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[adabeyta]

Sounds like a great idea, I'll take a look into moving query quantization to attention layer.

[jmkuebler]

Great thanks @adabeyta ! Note that for FA we did it in #24914

I just didn't get a chance to look into other backends yet (even to check which ones support quantized queries even).

So you help would be great to look into the other backends.