[Bug]: Qwen3.5-9B (BF16/AWQ) Illegal Memory Access in vLLM v0.17.0 (WSL2/RTX3090 Ti)

[d-etu]

Your current environment

Environment

• GPU: NVIDIA GeForce RTX 3090 Ti (24GB)
• WSL2: Ubuntu 24.04.4 LTS
• vLLM: 0.17.0
• PyTorch: 2.10.0
• CUDA: 12.8 (Driver 591.74)
• Python: 3.12.3

🐛 Describe the bug

I am encountering a torch.AcceleratorError: CUDA error: an illegal memory access was encountered (cudaErrorIllegalAddress) when initializing the Qwen/Qwen3.5-9B-AWQ model on vLLM v0.17.0.

Unlike previous reports suggesting this is limited to quantized models, I have confirmed that both Qwen/Qwen3.5-9B and cyankiwi/Qwen3.5-9B-AWQ-4bit models crash with the same error.

python3 -m vllm.entrypoints.openai.api_server
--model ./Qwen/Qwen3.5-9B-AWQ
--port 8000
--max-model-len 8192
--gpu-memory-utilization 0.9
--quantization compressed-tensors
--performance-mode throughput

Crash logs : https://gist.github.com/d-etu/8f406bf8c994737024194f1453e2ae7b

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[SaxenaShiv]

Hey, I am interested to work on this issue. Can you share the steps to reproduce the bug?

[d-etu]

Hey, I am interested to work on this issue. Can you share the steps to reproduce the bug?

Hi @SaxenaShiv, thanks for your interest in helping with this issue!

Here are the exact steps to reproduce the bug:

1. Environment Setup:

OS: WSL2 (Ubuntu 24.04.4 LTS)

Hardware: RTX 3090 Ti

Python: 3.12.3

Install vLLM 0.17.0 and PyTorch 2.10.0.

2. Model Preparation:
   You can test this with either the quantized model (cyankiwi/Qwen3.5-9B-AWQ-4bit) or the standard base model (Qwen/Qwen3.5-9B).
   Important note: The crash is not limited to quantized models; it happens on both AWQ and non-AWQ models.

3. Run the vLLM Server:
   Execute the following command (adjusting the --model path to your local directory or HF hub):

Bash
python3 -m vllm.entrypoints.openai.api_server
--model Qwen/Qwen3.5-9B-AWQ
--port 8000
--max-model-len 8192
--gpu-memory-utilization 0.9
--quantization compressed-tensors
--performance-mode throughput
(If you are testing the non-AWQ base model to reproduce the issue, simply remove the --quantization compressed-tensors flag).

Expected Behavior vs Actual Behavior:
Instead of initializing the server, the process will crash during the model weight loading phase with a torch.AcceleratorError: CUDA error: an illegal memory access was encountered (cudaErrorIllegalAddress).

Let me know if you need any specific stack traces from the crash logs or further details!

[SaxenaShiv]

Thanks for the comment! I have a question. Do I need to test on this specific model Qwen/Qwen3.5-9B-AWQ only? Can I use different model according to my current GPU hardware RTX 4060 with 8 GB VRAM?

[hocop]

I have the same crash with cuda 13.1 nvidia 590.48.01 on vllm==0.17.0 torch==2.10.0. Running QuantTrio/Qwen3.5-27B-AWQ on dual 3090

UPD: disabling mtp helped

[ZJY0516]

Could you please test whether this issue still exists on the main branch?

For issue #36249, I tested commit b7332b0 on main and didn't encounter any problems — it appears to have been fixed.

[d-etu]

Well, to track down the exact cause, I tried reinstalling a few things. It turns out that doing a clean wipe of my existing Studio Driver and installing the latest Game Ready Driver actually fixed the issue. It seems like my previous driver installation was somehow corrupted or tangled up.

So now, Qwen3.5-9B-AWQ works perfectly fine.

However... the base Qwen3.5-9B (BF16) is now failing to launch and throwing a completely new error that I haven't seen before. (I honestly have no idea why the AWQ version runs smoothly while the BF16 version crashes).

Crash logs : https://gist.github.com/d-etu/34c40c5502e7effc806a61eb85471fa7

[huaian2004]

vllm serve Qwen/Qwen3.5-27B --served-model-name vllm1 --reasoning-parser qwen3 -tp 2 --max-model-len 256K --enable-auto-tool-choice --tool-call-parser qwen3_coder --limit-mm-per-prompt '{"image":2,"video":0}' --speculative-config '{"method":"mtp","num_speculative_tokens":2}' --enable-chunked-prefill --max-num-seqs 1

I have the crash with vllm 0.17.0. If set "num_speculative_tokens" to 1, all good without any crash. It just happens when it's more than 1.

[ziliangqian]

Same issue, 0.17.0/0.17.1 serve Qwen/Qwen3.5-27B, If set "num_speculative_tokens" to 1, all good without any crash.
Crashed when num_speculative_tokens > 1.

[fangpings]

Same issue here with Qwen3.5-2B with mtp enabled in b200

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!