[core] fa4 support.

[sayakpaul]

What does this PR do?

Needs kernels to be installed from source for testing. Users might have to update nvidia-cutlass-dsl (or might even do a --force-reinstall).

In this PR, we're just supporting the bare minimum inference support. CP support can be shipped in a subsequent PR.

However, there's an error on H100. Code:

from diffusers import DiffusionPipeline 
import torch 

CKPT_ID = "black-forest-labs/FLUX.1-dev"

pipe = DiffusionPipeline.from_pretrained(
    CKPT_ID, torch_dtype=torch.bfloat16,
).to("cuda")
pipe.transformer.set_attention_backend("flash_4_hub")

prompt = """
cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California
highly detailed, high budget hollywood movie, cinemascope, moody, epic, gorgeous, film grain
"""

image = pipe(
    prompt,
    guidance_scale=3.5,
    num_inference_steps=50,
    generator=torch.manual_seed(0),
).images[0]
image.save("fa4.png")

Cc: @danieldk

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.