v3.0.2 - Direct Flash-Attention 2 Auto-Detection, Priority & Forge Kernel Proof Logging

v3.0.2 - Direct Flash-Attention 2 Auto-Detection, Priority & Forge Kernel Proof Logging

Summary:
v3.0.2 restores direct Flash-Attention 2 auto-detection, priority assignment, and execution visibility in A1111, adhering strictly to the Forge-Nunchaku (backend/attention_backend_info.py) kernel proof principles and A1111's direct-load architectural design (md/FA2_direct_load_design.md).

---

Highlights

Component / Area	Before (v3.0.1)	After (v3.0.2)
FA Direct Detection	Only attempted if --flash-attention CLI flag passed	Automatically detected and imported if flash_attn exists in environment
FA Optimizer Priority	priority = 50 (subordinate to Doggettx 90 & SDP 80/70)	priority = 110 (highest priority, above xformers 100, Doggettx 90, SDP 80/70)
xformers Coupling	is_available() restricted by not shared.xformers_available	Decoupled: Direct FA-2 available regardless of whether xformers is installed
Automatic Selection	Fell back to Doggettx / xformers unexpectedly	Selects Direct Flash-Attention by default unless explicit CLI flag given
Attention Kernel Proof Logging	Hardcoded unconditional print in process_images_inner	Forge-standard proof log emitted only after real CUDA kernel execution
Failure Observability	Silent fallback / obscured	Explicit notice [A1111] FA-2 kernel NOT used — ... fallback pytorch SDPA

---

1. Direct Flash-Attention 2 Auto-Detection (modules/sd_hijack_optimizations.py)

• Background: Upstream direct-load design allows Flash-Attention 2 (flash_attn_func) to execute directly without xformers. Previously, import was gated behind if shared.cmd_opts.flash_attention:, leaving FLASH_ATTN_AVAILABLE = False during standard launches without CLI arguments.
• Change: Directly attempts import flash_attn and resolves version/type (FA-2 or FA-3) on startup using module inspection helpers (_module_version, _module_file, _callable_file).
• Startup Console Output:

  [A1111] Flash-Attention direct import: FA-2 version <ver>
  

---

2. Optimizer Priority & Automatic Selection (modules/sd_hijack_optimizations.py & modules/sd_hijack.py)

• Background: In previous versions, SdOptimizationFlashAttn.priority was set to 50 and is_available() required not shared.xformers_available. When xformers was installed or when running with default settings, Doggettx (priority 90) or xformers took precedence, logging Applying attention optimization: Doggettx... done. during VAE/checkpoint switches.
• Change:
  • SdOptimizationFlashAttn.priority raised to 110.
  • Removed not shared.xformers_available restriction.
  • In modules/sd_hijack.py, apply_optimizations() honors explicit CLI arguments (e.g. --xformers, --opt-sdp-attention) if provided; otherwise, it selects the highest priority available optimizer (SdOptimizationFlashAttn at 110).
• Model Load Console Output:

  Applying attention optimization: flash-attention... done.
  

---

3. Forge-Standard Attention Kernel Proof Logging

• Background: Follows Forge-Nunchaku (backend/attention_backend_info.py) kernel proof rules:
  1. Never claim a kernel ran before it executes.
  2. Proof logs are emitted strictly after the real CUDA/Python kernel returns.
  3. Never print called for a PyTorch fallback.
  4. Output once per generation job.
• Implementation:
  • In flash_attention_forward() and flash_attn_attnblock_forward(), inspects the callable module and disk file location after flash_attn_func(...) completes.
  • Outputs proof line:

    [A1111] FA-2 kernel ran: flash_attn.flash_attn_interface.flash_attn_func ver=<version> file=<path>
    

  • If kernel execution encounters an error, logs honest fallback notice before switching to PyTorch SDPA:

    [A1111] FA-2 kernel NOT used — flash_attn_func failed (<exception>); fallback pytorch SDPA
    

---

4. Removal of Premature / False Logs (modules/processing.py)

• Background: modules/processing.py contained an unconditional print statement inside process_images_inner():

  print("[A1111] xformers.memory_efficient_attention called (FA-3→FA-2→Cutlass priority order)")

  This line executed on every generation job regardless of whether xformers, Doggettx, or flash-attention was active.
• Change: Removed the premature hardcoded log statement. Real execution proof is now delegated strictly to kernel return hooks.

---

Verification & Compatibility

• Python Version: CPython 3.14 (aligned with v3.0.0+ platform renewal).
• PyTorch / CUDA Stack: PyTorch 2.13.0+cu132 with Flash-Attention 2 2.8.4+cu132torch2.13.0.
• Fallback Guarantee: If flash_attn is unavailable or fails at runtime, execution cleanly degrades to SDPA then sub_quad_attention, maintaining zero generation aborts.