Temporarily Disable OpenMP support for libsox #1026
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Nov 13, 2020
mthrok
force-pushed
the
disable-openmp-sox
branch
7 times, most recently
from
b728dd5
to
44420fd
Compare
mthrok
force-pushed
the
disable-openmp-sox
branch
from
44420fd
to
6808cfa
Compare
mthrok
changed the title
vincentqb
approved these changes
Nov 18, 2020
vincentqb
reviewed
Nov 18, 2020
| @@ -76,5 +76,5 @@ ExternalProject_Add(libsox | |||
| DOWNLOAD_DIR ${ARCHIVE_DIR} | |||
| URL https://downloads.sourceforge.net/project/sox/sox/14.4.2/sox-14.4.2.tar.bz2 | |||
| URL_HASH SHA256=81a6956d4330e75b5827316e44ae381e6f1e8928003c6aa45896da9041ea149c | |||
| CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/build_codec_helper.sh ${CMAKE_CURRENT_SOURCE_DIR}/src/libsox/configure ${COMMON_ARGS} --with-lame --with-flac --with-mad --with-oggvorbis --without-alsa --without-coreaudio --without-png --without-oss --without-sndfile --with-opus | |||
| CONFIGURE_COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/build_codec_helper.sh ${CMAKE_CURRENT_SOURCE_DIR}/src/libsox/configure ${COMMON_ARGS} --with-lame --with-flac --with-mad --with-oggvorbis --without-alsa --without-coreaudio --without-png --without-oss --without-sndfile --with-opus --disable-openmp | |||
There was a problem hiding this comment.
Can you add a comment linking to this issue here?
There was a problem hiding this comment.
Good point. added the comment to this issue.
cpuhrsch
approved these changes
Nov 19, 2020
|
When we're bringing this back we need to figure out how to link against the version of openmp that pytorch is linking against. |
I am not fully aware of how to do this, but there is a similar conversation going in vision, this might be the way. |
mthrok
added a commit
to mthrok/audio
that referenced
this pull request
Nov 20, 2020
Currently `libsox` on Linux is compiled with GPU OpenMP and it interferes with the version PyTorch uses (Intel in case of binary distribution). This PR disables OpenMP support for `libsox`, while we investigate the way to use the same OpenMP as PyTorch's version.
mthrok
added a commit
that referenced
this pull request
Dec 3, 2020
Currently `libsox` on Linux is compiled with GPU OpenMP and it interferes with the version PyTorch uses (Intel in case of binary distribution). This PR disables OpenMP support for `libsox`, while we investigate the way to use the same OpenMP as PyTorch's version.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sox effect functions and subprocess cause issues like #1021.
In this PR, I added the tests that mimic this issue. This test properly fails in my development env (see the log bellow), however, it somehow does not fail in our CI. Furthermore, all the tests in
test/torchaudio_unittest/sox_effects/dataset_test.pycan fail with segmentation fault on my local env, but it does not happen in our CI. My environment is very similar to the CI settings (Docker/Ubuntu + Anaconda), and I can reproduce the issue on my working env but I have not figured out the key difference between my local env and CI.What are common to these failure cases include;
sox_effectsin subprocess.forkmethod.What we know is that;
forkmethod, settingOMP_NUM_THREDAS=1does not resolve the segmentation fault issue.spawnwould circumvent the segmentation fault.mp_context=multiprocessing.get_context("spawn")in ProcessPoolExecutor will crash lhotse-speech/lhotse#126OMP_NUM_THREADS=1helps the slowdown prepare.py slow speed k2-fsa/snowfall#18 but this affects all the operations of the application code (including PyTorch)libsoxresolves the issue. (forkworks too)-fopenmpflag when compilinglibsox.The above suggests that there is some sort of interference OpenMP of PyTorch (or MKL) and
libsox.This PR disables OpenMP support of
libsoxso that, at least users won't have an issue using these functionality while we investigate the issue further.error report
This one should go to
0.7.1release.