-
-
Notifications
You must be signed in to change notification settings - Fork 0
CUDA Setup and Troubleshooting
CUDA is A2M's NVIDIA-specific GPU path. A2M uses a managed CUDA ONNX Runtime pack, but compatible NVIDIA drivers, CUDA runtime libraries, and cuDNN must also be loadable.
For the current managed CUDA pack:
| Component | Current expectation |
|---|---|
| ONNX Runtime | 1.27.0 CUDA build |
| CUDA | 13.x |
| cuDNN | 9.x |
| Required DLL groups |
cudart64_13.dll, cublasLt64_13.dll, cudnn64_9.dll
|
These requirements are version-derived and can change with the runtime pack.
A2M checks the pack metadata, Python ABI, package layout, provider DLLs, and whether ONNX Runtime actually lists CUDAExecutionProvider. A helper process then attempts provider/model session validation. A folder full of CUDA DLLs is not by itself proof that inference can run.
The application discovers CUDA paths from environment variables and standard NVIDIA Toolkit directories. It also discovers cuDNN under the standard NVIDIA location and A2M's per-user dependency directory. A2M can download the configured cuDNN archive, verify its size and SHA-256, extract it safely, and add its bin directory to the current process path.
| Symptom | Likely cause | Verification / remedy |
|---|---|---|
| CUDA pack is offered repeatedly | Installed pack failed structural or provider validation | Retry only after checking the exact error; a failed staging pack is not activated |
| Required DLL is missing | CUDA Toolkit or cuDNN major does not match the pack | Compare the reported DLL name with the current requirements above |
| Provider is not exposed | Wrong or incomplete ONNX Runtime build | Re-downloading helps only if the published pack itself is correct |
| Provider load failed | Driver/library compatibility or DLL search path | Update the NVIDIA driver; verify CUDA/cuDNN are the required majors |
| Session creation failed | Provider loads but cannot run the model | Reduce variables, restart A2M, and capture the exact validation reason |
| A conversion fails and A2M returns to CPU | Recognized GPU failure | The failed file is not necessarily rerun; start it again in CPU mode |