[Feature]: Add EP/DP/PD deps in docker image

[simon-mo]

🚀 The feature, motivation and pitch

We should be adding DeepEP, pplx-kernels, DeepGEMM, and NIXL to the docker image.

You can refer to the tools/ep_kernels for instruction for first two. DeepGEEM repo.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[skirsten]

Hi, I am currently looking into this as I want to try EP+DP.

The DeepEP docs don't mention the patch file and instead say that its compatible with the upstream version 3.3.9. This version is also provided by nvidia-nvshmem-cu12.

So is the patch still needed or can we simply use the upstream version?

Edit: Nevermind, found this in the history:

**NOTE: After NVSHMEM v3.3.9, it is no longer necessary to apply our patch to achieve optimal performance.**

[simon-mo]

Contribution welcomed!

[XiaobinZhao]

+1

[skirsten]

Unfortunately I was not able to use the already installed nvidia-nvshmem-cu12 because it's missing files required for the build.

I was able to get all the software to work I think using this:

FROM local/vllm-openai:5bbaf492a6238ff517249e73151ae9989f7bea9e # build latest vllm

RUN mkdir /usr/local/nvshmem-3.3.9 && \
  wget -q https://developer.download.nvidia.com/compute/nvshmem/redist/libnvshmem/linux-x86_64/libnvshmem-linux-x86_64-3.3.9_cuda12-archive.tar.xz && \
  tar -xvf libnvshmem-linux-x86_64-3.3.9_cuda12-archive.tar.xz -C /usr/local/nvshmem-3.3.9 --strip-components=1 && \
  rm libnvshmem-linux-x86_64-3.3.9_cuda12-archive.tar.xz

ENV NVSHMEM_HOME=/usr/local/nvshmem-3.3.9

RUN --mount=type=cache,target=/root/.cache/uv \
  TORCH_CUDA_ARCH_LIST=9.0a+PTX uv pip install --system --no-build-isolation -v git+https://github.com/ppl-ai/pplx-kernels.git@1d76f488d794f01dc0e895cd746b235392379757

RUN --mount=type=cache,target=/root/.cache/uv \
  uv pip install --system --no-build-isolation -v git+https://github.com/deepseek-ai/DeepEP.git@4b67064d220f6f22619063a08a9fcec41f0c5614

# Note that this is before https://github.com/deepseek-ai/DeepGEMM/pull/112 which seems to break on H100:
# RuntimeError: Failed: CUDA runtime error csrc/jit/kernel_runtime.hpp:108 '98'
RUN --mount=type=cache,target=/root/.cache/uv \
  uv pip install --system --no-build-isolation -v git+https://github.com/deepseek-ai/DeepGEMM.git@03d0be3d2d03b6eed3c99d683c0620949a13a826

Unfortunately I was not yet able to test if it actually works. On 3x8xH100 nodes the Qwen/Qwen3-Coder-480B-A35B-Instruct dataparallel is failing with:

AssertionError: Hidden Size 6144 not in supported list of hidden sizes[2048, 2560, 4096, 5120, 7168]

Which should be fixed with #21818. I will test again when that is merged.

[jeejeelee]

Unfortunately I was not able to use the already installed nvidia-nvshmem-cu12 because it's missing files required for the build.

I was able to get all the software to work I think using this:

FROM local/vllm-openai:5bbaf492a6238ff517249e73151ae9989f7bea9e # build latest vllm

RUN mkdir /usr/local/nvshmem-3.3.9 &&
wget -q https://developer.download.nvidia.com/compute/nvshmem/redist/libnvshmem/linux-x86_64/libnvshmem-linux-x86_64-3.3.9_cuda12-archive.tar.xz &&
tar -xvf libnvshmem-linux-x86_64-3.3.9_cuda12-archive.tar.xz -C /usr/local/nvshmem-3.3.9 --strip-components=1 &&
rm libnvshmem-linux-x86_64-3.3.9_cuda12-archive.tar.xz

ENV NVSHMEM_HOME=/usr/local/nvshmem-3.3.9

RUN --mount=type=cache,target=/root/.cache/uv
TORCH_CUDA_ARCH_LIST=9.0a+PTX uv pip install --system --no-build-isolation -v git+https://github.com/ppl-ai/pplx-kernels.git@1d76f488d794f01dc0e895cd746b235392379757

RUN --mount=type=cache,target=/root/.cache/uv
uv pip install --system --no-build-isolation -v git+https://github.com/deepseek-ai/DeepEP.git@4b67064d220f6f22619063a08a9fcec41f0c5614

Note that this is before deepseek-ai/DeepGEMM#112 which seems to break on H100:

RuntimeError: Failed: CUDA runtime error csrc/jit/kernel_runtime.hpp:108 '98'

RUN --mount=type=cache,target=/root/.cache/uv
uv pip install --system --no-build-isolation -v git+https://github.com/deepseek-ai/DeepGEMM.git@03d0be3d2d03b6eed3c99d683c0620949a13a826
Unfortunately I was not yet able to test if it actually works. On 3x8xH100 nodes the Qwen/Qwen3-Coder-480B-A35B-Instruct dataparallel is failing with:

AssertionError: Hidden Size 6144 not in supported list of hidden sizes[2048, 2560, 4096, 5120, 7168]

Which should be fixed with #21818. I will test again when that is merged.

I have tested it recently and can fix this error, let's wait for #21818

[omerap12]

/cc
@jeejeelee I see that #21818 was merged. do you know if this #19653 (comment) should work now?