[Bug]: GPU build for SM-120/Blackwell uses old Pytorch (2.6) which is not compatible with Blackwell

[jhsmith409]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

RAGFlow workspace code commit ID

v0.20.0-60-g1bd64daf full cuda13-cu129

RAGFlow image version

v0.20.0-60-g1bd64daf full

Other environment information

Ubuntu 24 LTS, RTX 5090

Actual behavior

Issue Summary

The custom Dockerfile.cuda13 had two critical bugs that prevented proper CUDA 13.0 support for RTX 5090:

1. Missing Python interpreter files - The uv-installed Python interpreter was not copied to the production stage
2. PyTorch version mismatch - PyTorch cu130 nightly was installed in the builder stage but got overwritten by the cu126 version
   from uv.lock

Files Modified

/home/jhsmith/ragflow/Dockerfile.cuda13

Change 1: Added Python interpreter copy (Line 186)

Copy Python environment and packages

ENV VIRTUAL_ENV=/ragflow/.venv
COPY --from=builder ${VIRTUAL_ENV} ${VIRTUAL_ENV}
+COPY --from=builder /root/.local/share/uv/python /root/.local/share/uv/python
ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"

Problem: The venv contained symlinks to Python at /root/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/bin/python3.10,
but this directory wasn't copied to the production stage, causing ModuleNotFoundError: No module named 'beartype' at runtime.

Change 2: Moved PyTorch cu130 installation to production stage (Lines 189-192)
ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"

+# Install PyTorch cu130 nightly in production stage to ensure it's not overwritten
+# This must be done after copying venv to override the cu12 version from uv.lock
+RUN --mount=type=cache,id=ragflow_uv_cuda13_prod,target=/root/.cache/uv,sharing=locked \

• uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu130

ENV PYTHONPATH=/ragflow/

Problem: The builder stage installed PyTorch cu130 nightly at line 157-159, but when the venv was copied to production, the
uv.lock file's pinned torch 2.7.1+cu126 version remained. This version doesn't support Blackwell architecture (sm_120/compute
capability 12.0), causing the RTX 5090 to be unable to compile GPU kernels.

Verification

After fixes:

• PyTorch: 2.10.0.dev20251008+cu130 (was 2.7.1+cu126)
• CUDA: 13.0 (was 12.6)
• Compute capability: 12.0 (Blackwell/sm_120 now supported)
• Successfully runs GPU matrix operations on RTX 5090

Root Cause

The Dockerfile.cuda13 was likely copied from the standard Dockerfile but the PyTorch override strategy didn't account for:

1. Multi-stage builds where the uv Python installation directory needs explicit copying
2. The uv.lock file pinning torch dependencies that get reinstalled when copying the venv between stages

Suggested Fix for Repository

The standard Dockerfile doesn't have these issues, suggesting Dockerfile.cuda13 was an incomplete adaptation. Consider either:

1. Applying these same fixes to Dockerfile.cuda13 in the repository
2. Documenting that users need PyTorch cu130+ for Blackwell GPUs

Expected behavior

I expected the code to compile/install without error.

Steps to reproduce

Install with gpu support for Blackwell.  Modify .env per instructions.

Additional information

No response

[jhsmith409]

Further complication seems to be ONNX Runtime doesn't support CUDA 13.0 yet (or it has to be re-built from source).

[KevinHuSh]

The ONNX Runtime has some issues running with CUDA. So, running on CPU only is strongly recommanded.