[Feature] Upgrade to ONNX Runtime 1.27.0 for official CUDA 13 support #29491
altheali533
started this conversation in
Feature Request
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I have searched the existing feature requests, both open and closed, to make sure this is not a duplicate request.
The feature
I have searched the existing feature requests to make sure this is not a duplicate request.
Yes
The feature
ONNX Runtime v1.27.0 was just released and it's a significant milestone for CUDA 13 support:
onnxruntime-<os>-<arch>-gpu_cuda13-<version>onnxruntime-gpu1.27.0 wheels for CUDA 13 are already live on PyPIRelease notes: https://github.com/microsoft/onnxruntime/releases/tag/v1.27.0
Previously, the blocker for Immich adding CUDA 13 support was that onnxruntime only had CUDA 13 as an unstable nightly build (no official stable release). That blocker is now resolved.
Immich's
machine-learning/Dockerfilecurrently builds the CUDA image fromnvidia/cuda:12.2.2-runtime-ubuntu22.04, which causes failures for users running newer NVIDIA drivers that only ship CUDA 13 (see #21910, #21911, and the arm64/DGX Spark discussion #10647). Many users are currently forced to either downgrade their host driver to a CUDA 12.x-compatible version or patch a custom Dockerfile themselves.Describe the solution you'd like
Upgrade the
machine-learningservice to onnxruntime 1.27.0 and switch the-cudaimage's base to a CUDA 13 runtime image (e.g.nvidia/cuda:13.x-runtime-ubuntu24.04or similar), following the same approach as the recent ONNX Runtime bump PR (#23458).This would:
LD_LIBRARY_PATHworkaroundsDescribe alternatives you've considered
Currently working around this by pinning host NVIDIA driver/CUDA to a 12.x-compatible version, or building a custom machine-learning image from source. Neither is ideal for a "batteries included" self-hosted deployment.
Additional context
Platform
All reactions