Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

could not create a dilated convolution forward descriptor, in file tensorflow/core/kernels/mkl_conv_ops.cc:1111 #27906

Closed
nm46nm opened this issue Apr 17, 2019 · 11 comments
Assignees
Labels
comp:mkl MKL related issues stat:awaiting response Status - Awaiting response from author

Comments

@nm46nm
Copy link

nm46nm commented Apr 17, 2019

When I tried to run some yolov3 codes, I always got the following error:
`Traceback (most recent call last):

File "", line 1, in
runfile('C:/Users/10137/Desktop/learn/python/yolov3_keras_qwe/train.py', wdir='C:/Users/10137/Desktop/learn/python/yolov3_keras_qwe')

File "C:\Users\10137\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)

File "C:\Users\10137\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "C:/Users/10137/Desktop/learn/python/yolov3_keras_qwe/train.py", line 190, in
_main()

File "C:/Users/10137/Desktop/learn/python/yolov3_keras_qwe/train.py", line 65, in _main
callbacks=[logging, checkpoint])

File "C:\Users\10137\Anaconda3\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)

File "C:\Users\10137\Anaconda3\lib\site-packages\keras\engine\training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)

File "C:\Users\10137\Anaconda3\lib\site-packages\keras\engine\training_generator.py", line 217, in fit_generator
class_weight=class_weight)

File "C:\Users\10137\Anaconda3\lib\site-packages\keras\engine\training.py", line 1217, in train_on_batch
outputs = self.train_function(ins)

File "C:\Users\10137\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 2715, in call
return self._call(inputs)

File "C:\Users\10137\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 2675, in _call
fetched = self._callable_fn(*array_vals)

File "C:\Users\10137\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1439, in call
run_metadata_ptr)

File "C:\Users\10137\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 528, in exit
c_api.TF_GetCode(self.status.status))

AbortedError: Operation received an exception:Status: 3, message: could not create a dilated convolution forward descriptor, in file tensorflow/core/kernels/mkl_conv_ops.cc:1111
[[{{node conv2d_2/convolution}}]]`

@ianlokh
Copy link

ianlokh commented Apr 18, 2019

I am also getting the same error on TensorFlow 1.13.1. There appears to be a similar issue (#24650) that was to be resolved in 1.13.1 but the error line was different in that case.

@ppwwyyxx
Copy link
Contributor

#24650 was not resolved in 1.13.1

@nm46nm
Copy link
Author

nm46nm commented Apr 18, 2019

I downgraded the keras to 2.1.5 and the error didn't occur again. This may help.

@ianlokh
Copy link

ianlokh commented Apr 18, 2019

Hi nm46nm, thanks it worked but TensorFlow got downgraded to 1.9. I will try to apply TF 1.12 and see if it works noting that this was a non-issue in the latest master branch (thanks Yuxin)

@achandraa
Copy link

@nm46nm and @ianlokh : Is this resolved? Please let us know. Thanks!

@achandraa achandraa added stat:awaiting response Status - Awaiting response from author comp:mkl MKL related issues labels Apr 26, 2019
@rbgt
Copy link

rbgt commented Apr 29, 2019

Downgrading to tensorflow 1.9 worked fine for me

@achandraa
Copy link

That's great. We will be closing the issue since it is resolved. Thanks!

@ianlokh
Copy link

ianlokh commented Apr 30, 2019

Hi achandraa, yes downgrading to TF 1.9 resolved my issue. As a side note, I upgraded TF to 1.12 and it worked as well.

@nm46nm
Copy link
Author

nm46nm commented May 1, 2019

@nm46nm and @ianlokh : Is this resolved? Please let us know. Thanks!

I just use the version 2.1.5 in which there is no error. Haven't tried new version again.

crdietrich added a commit to crdietrich/cnn_profiler that referenced this issue May 5, 2019
CPU bug in tensorflow 1.13.1, fixed in 2.0 alpha, chose to downgrade to avoid API changes.  See tensorflow/tensorflow#27906
@mengxia1994
Copy link

According to commits above, i tried to downgrade tensorflow from 1.13 to 1.12 and it also worked for me.

@Isaryll
Copy link

Isaryll commented Aug 27, 2019

I tried to update tensorflow from 1.13.1 to 1.14 and it work for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:mkl MKL related issues stat:awaiting response Status - Awaiting response from author
Projects
None yet
Development

No branches or pull requests

7 participants