New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
UNIMPLEMENTED: DNN library is not found. #10590
Comments
This is a strange error related to XLA gpu? @cheshire could you help take a look? |
@saberkun this is a cuDNN error, which is used for convolutions in both TF and XLA, it's not specific to XLA:GPU in any way. Probably cuDNN is not linked properly. |
Hi @MarkosMuche Can you take a look at the workaround proposed in this link and see if it helps in resolving your issue? Thanks! |
@pindinagesh I tried it; however It is throwing another error. AssertionError: Duplicate registrations for type 'experimentalOptimizer' . I searched for solutions and found out that it is tensorflow and keras version. |
unfortunately, the above issue (tensorflow keras compatibility) couldn't be solved. Is there any other way to solve the DLL issue without going back to tensorflow 2.7? |
Can you make sure the keras version is also 2.8? For keras experimental optimizer issue, @chenmoneygithub |
I am having the same issue, has there been a resolution? The CNN model runs in jupyter notebook but not in colab. |
What's the tensorflow version and cuda used by the colab runtime? |
Hi Saberkun, I followed the instructions on googlecolab/colabtools#2600 and used:
and this resolved the issue. |
@MMBB7 thank you very much! Let's highlight the solution you found. |
under cuda 11.2 install cudnn>8.2 |
@MMBB7 thank you .. this worked for me |
that worked |
Hi @MMBB7 As I am using kerasCV for augmentation, it needs Tensorflow 2.9+ version, so switching to Tensorflow 2.8 will not work in my use case. Any workaround or solution for this? |
What is the issue? I am running the models on Google Colab. Yesterday it was running/working fine but today I am running the notebooks, it is giving me this error. |
Number 3, solved my issue, thanks. |
!apt install --allow-change-held-packages libcudnn8=8.1.0.77-1+cuda11.2 |
Thanks it solved the error. |
It works for me too ! thx |
Great. |
how can i install this for pip packages , bcz i have same error in jupyter notebook |
This is the correct solution however I had to add
to get this to work. Also I did not touch the tensorflow installation. only the install and PATH changes related to libcudnn Hope this is helpful to other struggling with this issue. |
I was following the movinet transfer learning tutorial: https://github.com/tensorflow/models/blob/master/official/projects/movinet/movinet_tutorial.ipynb
I downloaded it, uploaded it to colab and run it. I am encountering the problem "UNIMPLEMENTED: DNN library is not found." It doesn't run on GPU.
Node: 'movinet_classifier_1/movinet/stem/stem/conv3d/StatefulPartitionedCall'
Failed to determine best cudnn convolution algorithm for:
%cudnn-conv = (f32[8,8,86,86,8]{3,2,1,4,0}, u8[0]{0}) custom-call(f32[8,8,173,173,3]{3,2,1,4,0} %pad, f32[1,3,3,3,8]{2,1,0,3,4} %copy.1), window={size=1x3x3 stride=1x2x2}, dim_labels=b012f_012io->b012f, custom_call_target="__cudnn$convForward", metadata={op_type="Conv3D" op_name="Conv3D" source_file="/usr/local/lib/python3.7/dist-packages/keras/layers/convolutional/base_conv.py" source_line=232}, backend_config="{"conv_result_scale":1,"activation_mode":"0","side_input_scale":0}"
Original error: UNIMPLEMENTED: DNN library is not found.
To ignore this failure and try to use a fallback algorithm (which may have suboptimal performance), use XLA_FLAGS=--xla_gpu_strict_conv_algorithm_picker=false. Please also file a bug for the root cause of failing autotuning.
[[{{node movinet_classifier_1/movinet/stem/stem/conv3d/StatefulPartitionedCall}}]] [Op:__inference_train_function_104591]
Solution found:
@MMBB7 followed the instructions on googlecolab/colabtools#2600 and used:
factory reset runtime in colab runtime
!pip install tensorflow==2.8
!apt install --allow-change-held-packages libcudnn8=8.1.0.77-1+cuda11.2
The text was updated successfully, but these errors were encountered: