could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR

[peternara]

Hi, @argman

I have a next problem (training )

....
2017-08-31 18:22:46.750620: E tensorflow/stream_executor/cuda/cuda_dnn.cc:371] could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2017-08-31 18:22:46.750670: E tensorflow/stream_executor/cuda/cuda_dnn.cc:338] could not destroy cudnn handle: CUDNN_STATUS_BAD_PARAM
2017-08-31 18:22:46.750683: F tensorflow/core/kernels/conv_ops.cc:672] Check failed: stream->parent()->GetConvolveAlgorithms( conv_parameters.ShouldIncludeWinogradNonfusedAlgo(), &algorithms)

Do you know this problem? (I used 2 gpu)

[argman]

@peternara , I think its caused by process not correctly stopped, you can check via ps -aux|grep python

[peternara]

@argman thank for rely.
This case is when a segment fault occurs. A python process exists, but some processes are dead. So, no model was created. I am only guessing a memory problem between cuda and tf.

[ashleylid]

I am also having this issue. There are no related python processes running.

[flifuehu]

Run sudo rm -rf ~/.nv/ to fix it.

[burakekincioglu]

@flifuehu what does this code do? sudo rm -rf ~/.nv/

[HuangShiqing]

@burakekincioglu
i think the code is to delete a file named ".nv" in your home directory.It may be a nvidia configuration file.
i think the reason it can resolve the problem of some guy is restart the configuration.
Anyway,it doesn't work for me