-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Using onnxruntime-genai-directml without CUDA #555
Comments
Although the error reports that the CUDA library is not found, the real problem is that one of the dependency DLL was not found:
In the event of a failed onnxruntime_genai import (with the missing DLL error), ort-genai assumes that the installed package is the CUDA one and the missing DLL error is from Python not being able to find the CUDA DLLs. In such a scenario, ORT GenAI tries to add a search path for the missing CUDA DLL before erroring out if still not found. Unfortunately, the error is misleading when the missing DLL is some other DLL (which seems to be the case for you). Are you able to identify the missing DLL? Could you share the output of this command?
Please replace the |
Here is the output of Dump of file <snip>\envs\phi3\lib\site-packages\onnxruntime_genai\onnxruntime_genai.cp312-win_amd64.pyd
File Type: DLL
Image has the following dependencies:
onnxruntime.dll
python312.dll
d3d12.dll
dxgi.dll
KERNEL32.dll
MSVCP140.dll
CONCRT140.dll
VCRUNTIME140_1.dll
VCRUNTIME140.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-filesystem-l1-1-0.dll
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-locale-l1-1-0.dll
api-ms-win-crt-environment-l1-1-0.dll |
We've heard some others have also faced this issue. Although I haven't been able to reproduce it. Some of those who were able to reproduce this mentioned that the missing lib was zlib.dll. I will find a machine where I can reproduce this and fix it. Thanks for sharing the details. |
@mmaitre314, it turns out to be that conda uses old vc runtime and it is not compatible with new visual studio compiler. Could you please try run "conda install conda-forge::vs2015_runtime" to upgrade your vc runtime version. We are working on a hot fix on it too. |
Some more context: MSVC team recently updated their stl library and we compile our source code in onnxruntime-genai against the updated stl library. But the conda env vc runtime dlls are older than the expected version. This causes the problem What can be done to address this? In due course, conda dlls will be updated. To address this on our end, we added the PR #599 so as to extend the support to a slightly older vc runtime dll version. But this won't help if the vc runtime dll is even older. This will be a part of the 0.3.0 ort-genai release. An alternative is to use the command @yufenglee shared and update the vc runtime version Please give it a try and let us know how that goes. I'll close this issue now. Do let us know if you encounter more issues. |
Confirming installing vs2015_runtime resolved the issue. |
Also confirming the following command solved the issue for me too.
|
I pip installed the DirectML version of onnxruntime-genai (onnxruntime-genai-directml==0.3.0rc2) hoping to avoid having to install CUDA but importing the library fails mentioning there is no CUDA. Is there a way to use ONNX with DirectML and not CUDA?
The text was updated successfully, but these errors were encountered: