Add LLM support for cuda backend

[larryliu0820]

Summary

This PR extends CUDA support for text-only LLM workflows and adds CI coverage for Qwen3-0.6B artifacts and pybind execution.

Why

We already validate CUDA multimodal paths, but text-generation CUDA coverage (especially Qwen3) was incomplete.
This change adds export/run support and CI wiring so CUDA text-generation artifacts are exercised in automated tests.

What changed

CUDA LLM runner/build support

• Added llama-cuda and llama-cuda-debug Makefile targets.
• Added CUDA presets/workflow presets in examples/models/llama/CMakePresets.json.
• Updated examples/models/llama/CMakeLists.txt to link CUDA backend when EXECUTORCH_BUILD_CUDA=ON.
• Updated examples/models/llama/main.cpp:
  • Added --data_path convenience flag (single PTD path).
  • Added --prompt_file support for file-based prompts.

Gemma3 runner usability

• Updated examples/models/gemma3/e2e_runner.cpp:
  • Added --max_new_tokens.
  • Added --stop_sequence early-stop behavior.

Optimum exporter integration and CI pin

• Bumped optimum-executorch CI pin to:
  • a9592258daacad7423fd5f39aaa59c6e36471520
• Added Qwen/Qwen3-0.6B handling in .ci/scripts/export_model_artifact.sh for text-generation.

HuggingFace optimum CUDA test path

• Updated .ci/scripts/test_huggingface_optimum_model.py (test_text_generation):
  • Supports recipe=cuda export (--device cuda --dtype bfloat16).
  • Supports CUDA quantization for this path:
    • --qlinear 4w
    • --qlinear_packing_format tile_packed_to_4d
    • --qembedding 8w
  • Validates presence of aoti_cuda_blob.ptd.
  • Passes blob path into TextLLMRunner.

CUDA workflow updates

• Updated .github/workflows/cuda.yml:
  • Added Qwen/Qwen3-0.6B to CUDA export matrix.
  • Updated test-cuda-pybind matrix to explicit artifact mapping.
  • Added Qwen non-quantized and quantized-int4-tile-packed artifact runs in pybind test.
  • Switched download-artifact to matrix-provided artifact name.

Validation

Rely on new CI jobs.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17316

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures, 8 Cancelled Jobs, 6 Unrelated Failures

As of commit 870cdc6 with merge base 1f4ad07 ():

NEW FAILURES - The following jobs have failed:

• cuda-perf / upload-benchmark-results (gh)
  Process completed with exit code 255.
• pull / test-multimodal-linux (gemma3-4b) / linux-job (gh)
  RuntimeError: Command docker exec -t c6eda1a4cc6ad39291b471ea920792d653834f86dfedb51744c73bb943419573 /exec failed with exit code 139
• pull / unittest-arm-backend-with-no-deps (test_pytest_ops_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 7713a58c672bb12d2d5a3f075144d9e5de346e18d4e377ebd86b53d8bbab249f /exec failed with exit code 1
• Test Metal Backend / export-model-metal-artifact (mistralai, Voxtral-Mini-3B-2507, non-quantized) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• Test Metal Backend / export-model-metal-artifact (mistralai, Voxtral-Mini-3B-2507, quantized-int4-metal) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• Test Metal Backend / export-model-metal-artifact (openai, whisper-large-v3-turbo, non-quantized) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• Test Metal Backend / export-model-metal-artifact (openai, whisper-small, non-quantized) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• pull / test-lora-linux / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  ##[error]The operation was canceled.
• trunk / test-demo-backend-delegation (buck2) / linux-job (gh)
• trunk / test-demo-backend-delegation (cmake) / linux-job (gh)
• trunk / test-models-linux-aarch64 (phi_4_mini, portable, linux.arm64.m7g.4xlarge) / linux-job (gh)
  ##[error]The operation was canceled.
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh)

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• cuda-perf / benchmark-cuda (nvidia/parakeet-tdt, non-quantized, nvidia_parakeet-tdt, 50) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-windows (mobilebert, xnnpack-q8) / windows-job (gh) (detected as infra flaky with no log or failing log classifier)
• trunk / test-models-windows (vit, xnnpack-q8) / windows-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (mistralai, Voxtral-Mini-3B-2507, non-quantized) / windows-job (gh) (trunk failure)
  Process completed with exit code 1.
• Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (mistralai, Voxtral-Mini-3B-2507, quantized-int4-weight-only) / windows-job (gh) (trunk failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[lucylq]

why do we need this? Can we just have data_paths above?

[larryliu0820]

oh. can be removed, the argument is slightly different from the other runners (data_path vs data_paths) but I guess it's not good to have both.

[lucylq]

Ahh I should remove data_path. All the APIs support data_paths now. Should be experimental so don't need to deprecate

[Gasoonjia]

why do we need to update gemme3 runner to support qwen3 with pybinding?

[larryliu0820]

not needed. I can put this into another PR