[Feature]: DeepSeek-V4 Flash sm_80 (A100/A800) support

[xhd0728]

🚀 The feature, motivation and pitch

I am trying to run DeepSeek-V4-Flash on NVIDIA A800 GPUs, which are sm_80 / Ampere GPUs. Currently, using vllm/vllm-openai:deepseekv4-cu129, the model fails during initialization with:

RuntimeError: Assertion error (.../deepgemm-src/csrc/apis/hyperconnection.hpp:56): Unsupported architecture

This still happens with --enforce-eager, so it seems related to the DeepGEMM / HyperConnection kernels rather than torch.compile.

Are there any plans to support sm_80 GPUs such as A100/A800 for DeepSeek-V4, possibly with a slower fallback path? A100/A800 GPUs are still widely used in many research and production clusters, so even a functional fallback would be very helpful.

Alternatives

The current alternatives seem to be:

1. Use newer GPUs such as H100/H200/B200.
2. Wait for a backend/kernel fallback that supports sm_80.

I am not sure whether sm_80 support is planned or whether it should currently be considered unsupported for this model.

Additional context

Environment:

• GPU: NVIDIA A800, sm_80 / Ampere
• vLLM image: vllm/vllm-openai:deepseekv4-cu129
• Model: DeepSeek-V4-Flash
• Single node, 8 GPUs

Simplified command:

docker run -d --gpus all \
  -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
  -v /path/to/models:/workspace \
  -p 39001:8000 \
  --ipc=host \
  vllm/vllm-openai:deepseekv4-cu129 \
  /workspace/DeepSeek-V4-Flash \
  --served-model-name dsv4-flash \
  --dtype bfloat16 \
  --kv-cache-dtype fp8 \
  --enable-expert-parallel \
  --data-parallel-size 4 \
  --enforce-eager

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Cancerhzc]

Hi @xhd0728 ! May I ask if you deployed it on 8 × NVIDIA A800 (40 GB) GPUs? I’m also planning to deploy on sm_80 / Ampere (A100), and I’d like to consult you about your GPU memory usage during runtime.

[xhd0728]

Hi @Cancerhzc! I just tried deploying DSV4-Flash on 8 × NVIDIA A800 80G GPUs, but it seems that the current implementation does not support the A800 architecture.

Here is the full Docker command I used:

docker run -d --gpus all \
  -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
  -e OMP_NUM_THREADS=1 \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \
  -v /path/to/models:/workspace \
  -p 39001:8000 \
  --ipc=host \
  --name dsv4-flash \
  vllm/vllm-openai:deepseekv4-cu129 \
  /workspace/DeepSeek-V4-Flash \
  --served-model-name dsv4-flash \
  --host 0.0.0.0 \
  --port 8000 \
  --api-server-count 1 \
  --dtype bfloat16 \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --enable-expert-parallel \
  --data-parallel-size 4 \
  --tokenizer-mode deepseek_v4 \
  --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v4 \
  --speculative_config '{"method":"mtp","num_speculative_tokens":2}' \
  --gpu-memory-utilization 0.9 \
  --trust-remote-code \
  --enforce-eager \
  --log-error-stack

I also tried adding/removing the following options, but neither seemed to resolve the issue:

```shell
--kv-cache-dtype fp8
--enforce-eager
--compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE", "custom_ops":["all"]}'

[RandyChen1985]

The A800 doesn't support FP8 or FP4. It can't even deploy GLM-5 because its architecture is no longer compatible.

[feria-tu]

The A800 doesn't support FP8 or FP4. It can't even deploy GLM-5 because its architecture is no longer compatible.

But I used A800 to deploy minimax-m2.7, which is also fp8.

[RandyChen1985]

The A800 doesn't support FP8 or FP4. It can't even deploy GLM-5 because its architecture is no longer compatible.

But I used A800 to deploy minimax-m2.7, which is also fp8.

DSA kernel dependency: GLM-5's DSA is based on FlashMLA, which only supports SM90 (Hopper / H100) and above. A800 does not have the optimized operators.

[haosdent]

GLM-5's DSA is based on FlashMLA, which only supports SM90 (Hopper / H100) and above

I implemented a Trition MLA for GLM 5.1 #38476

[zuuky]

hi @haosdent , Does your implementation of Triition MLA also work with deepseek v4?

[haosdent]

Does your implementation of Triition MLA also work with deepseek v4?

By right yes, but it only addresses the MLA issue; it still not sufficient to run Deepseek-v4 flash due to other gaps like CSA/HCA