[Feature]: Batch-invariant support for GDN_ATTN (Qwen3-Next / Qwen3.6 hybrid Mamba+GDN MoE models)

[timbmg]

Your current environment

Versions

• vLLM: 0.21.0 (Docker image vllm/vllm-openai:v0.21.0) and nightly (digest sha256:d1bd760bf6630f67378206c7945afb6ab9bc046064a51fe421461e91261dcd7b, pulled 2026-05-18)
• PyTorch: 2.11.0+cu130
• CUDA: 13.0
• GPU: NVIDIA A100-SXM4-80GB (compute capability 8.0, SM80)
• Model: cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit (Qwen3-Next-style hybrid Mamba + Gated-Delta-Net + softmax-attention MoE; quantization: compressed-tensors)
• TP: 1

🐛 Describe the bug / 🛠 Feature request

Setting VLLM_BATCH_INVARIANT=1 on a model that contains GDN (Gated-Delta-Net) linear-attention layers causes engine startup to abort with:

RuntimeError: VLLM batch_invariant mode is not supported for GDN_ATTN.

Source: vllm/v1/attention/selector.py:154 in _cached_get_mamba_attn_backend.

This is a hard incompatibility — no fallback, no partial mode. It blocks reproducibility work for all Qwen3-Next / Qwen3.6-style models (and any other hybrid Mamba/GDN architecture).

Reproduction

docker run --rm --gpus all --ipc host \
  -e HUGGING_FACE_HUB_TOKEN=... \
  -e VLLM_BATCH_INVARIANT=1 \
  vllm/vllm-openai:v0.21.0 \
  --model cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit \
  --trust-remote-code \
  --max-model-len 20480

Both v0.21.0 and nightly (May 2026) fail with the same error. The check is triggered as soon as the engine selects the Mamba/GDN attention backend during init, before any AWQ-kernel logic runs — so it is independent of --quantization, --attention-backend, VLLM_ATTENTION_BACKEND (unrecognized in 0.21.0), and other workaround knobs.

Related

• [Feature] Support compile mode for batch invariance on SM80 #42456 — added SM80 batch-invariant support for the regular attention path. This issue requests the analogous coverage for the linear-attention (Mamba/GDN) path.
• [Bug]: Batch Invariant didn't work with Qwen3-30B-A3B-Instruct-2507-AWQ #29581 — closed; addressed batch-invariance for AWQ-Marlin kernels on the non-hybrid Qwen3-30B-A3B. Different layer family (no GDN); doesn't cover this case.
• [Bug]: Batch invariance fails on NVIDIA B200 (Blackwell) with torch.compile #32992 — closed; B200 / Blackwell + torch.compile. Different hardware.

Happy to test patches on A100 + Qwen3.6-A3B if helpful.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[timbmg]

Also appears in Qwen3.5. But in vllm == 0.19, it can be used by set attention backend. I don't think it's a proper solution, but it works... Looking forward for an official support.

You mean there is some batch invariant attention mechanism for Qwen 3.5/3.6 MoE models?

[Liaokc]

Also appears in Qwen3.5. But in vllm == 0.19, it can be used by set attention backend. I don't think it's a proper solution, but it works... Looking forward for an official support.

You mean there is some batch invariant attention mechanism for Qwen 3.5/3.6 MoE models?

I'm not sure it's a right way to use... Qwen3.6-35B-A3B can be set into batch invariant in vllm == 0.19 as follows:

VLLM_BATCH_INVARIANT=1 nohup vllm serve /home/sound/data/checkpoints/Qwen/Qwen3.6-35B-A3B/ --port 8000 --tensor-parallel-size 2 --attention-backend FLASH_ATTN --max-model-len 262144 --reasoning-parser qwen3
It can keep batch invariant in my picture QA task

There might be an issue with your parameters, as VLLM_BATCH_INVARIANT=1 hasn't taken effect. I checked the source code, and it indicates that when your environment variables include both VLLM_BATCH_INVARIANT and GDN_attention, a check is made for mamba_attn_backend.supports_batch_invariance(). Currently, it is not supported. Regarding batch invariance, it might be because your inference batch_size is set to 1, which is why the inference results are consistent. This is my speculation, but you can verify it...