New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ffmpeg.decode_audio cannot be run in parallel #5804
Comments
@fredbertsch You are listed as owner for this contrib dir... could you please take a look? |
I also have this problem. Will you fix this? |
@fredbertsch Any updates here? |
Just keeping folks in the loop, we have a candidate fix internally that @fredbertsch authored. Hopefully it will show up in master over the next day or two! |
A test and fix were added internally, and they should propagate here soon. |
Bug is at: tensorflow#5804 Fix is to add a unique identifier to each temp file name. The id is unique to the process. Multiple processes could still have a conflict, though even there the odds do go down somewhat with this fix. PiperOrigin-RevId: 173261202
It has been 14 days with no activity and this issue has an assignee.Please update the label and/or status accordingly. |
To decode the input, the function writes the content to a temporary file. Its name is generated by the function
GetTempFilename
found intensorflow/contrib/ffmpeg/default/ffmpeg_lib.cc
. The template for the filename is%tmp_dir/tmp_file_%PID.%EXT
.When using multiple decoders in parallel this causes an undetermined behaviour since all decoders want to write and afterwards delete the same file.
A possible solution would be to use the thread id instead of the process id. I.e.
The first two lines are necessary because glibc does not wrap the call.
This solution works for me (on Linux). I'm, however, not sure if it works on all supported platforms. If that's fine, I can make a pull request.
The text was updated successfully, but these errors were encountered: