[FA4] Add kernels fallback

[vasqu]

Depends on #44887 and kernels being version 12.3

Works OOB with little changes! Example script for demonstration:

from transformers import AutoModelForCausalLM, AutoTokenizer


fa_version = 4
#model_id = "openai/gpt-oss-20b"
model_id = "meta-llama/Llama-3.2-3B-Instruct"


model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    attn_implementation=f"flash_attention_{fa_version}",
).eval()
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
inputs = tokenizer(
    ["Hello, how are you?", "How are you? Tell me the name of the president of"],
    padding=True,
    padding_side="left",
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=50, do_sample=False)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

niice