Undefined references when building liblame

[mmxgn]

🐛 Bug

Hello all,
Thanks for maintaining torchaudio.

When trying to do python setup.py install from within anaconda it fails with error code 2 at the stage when building liblame. make.log in third_party/tmp/lame-3.99.5 show errors similar to:
undefined reference to `tgetnum'
(see below for full list).

To Reproduce

Steps to reproduce the behavior:

1. Activate anaconda3 environment
2. cd to the root audio directory
3. python setup.py install

Making all in mpglib
Making all in libmp3lame
Making all in i386
Making all in vector
Making all in frontend
libtool: link: /home/mmxgn/anaconda3/envs/tf_gpu/bin/x86_64-conda_cos6-linux-gnu-cc -Wall -pipe -fPIC -I/home/mmxgn/anaconda3/include -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z -Wl,relro -Wl,-z -Wl,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath -Wl,/home/mmxgn/anaconda3/lib -Wl,-rpath-link -Wl,/home/mmxgn/anaconda3/lib -o lame lame_main.o main.o brhist.o console.o get_audio.o lametime.o parse.o timestatus.o  -L/home/mmxgn/anaconda3/lib ../libmp3lame/.libs/libmp3lame.a -lncurses /home/mmxgn/anaconda3/lib/libiconv.so -lm -Wl,-rpath -Wl,/home/mmxgn/anaconda3/lib -Wl,-rpath -Wl,/home/mmxgn/anaconda3/lib
/home/mmxgn/anaconda3/envs/tf_gpu/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: console.o: in function `get_termcap_string.constprop.4':
console.c:(.text+0x1d): undefined reference to `tgetstr'
/home/mmxgn/anaconda3/envs/tf_gpu/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: console.o: in function `apply_termcap_settings.constprop.3':
console.c:(.text+0x80): undefined reference to `tgetent'
/home/mmxgn/anaconda3/envs/tf_gpu/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: console.c:(.text+0xa0): undefined reference to `tgetnum'
/home/mmxgn/anaconda3/envs/tf_gpu/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: console.c:(.text+0xc1): undefined reference to `tgetnum'
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:358: lame] Error 1
make[1]: *** [Makefile:349: all-recursive] Error 1
make: *** [Makefile:276: all] Error 2

Expected behavior

To build liblame succesfully and continue installation from there.

Environment

• What commands did you used to install torchaudio (conda/pip/build from source)?

conda create -n pytorch python=3.7
conda activate pytorch
conda install -c conda-forge sox
python setup.py install

• If you are building from source, which commit is it?

da59f59

• What does torchaudio.__version__ print? (If applicable)
  Not applicable

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

PyTorch version: 1.5.0
Is debug build: No
CUDA used to build PyTorch: 10.2

OS: Arch Linux
GCC version: (GCC) 10.1.0
CMake version: version 3.17.2

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GTX 1080 Ti
Nvidia driver version: 440.82
cuDNN version: /usr/lib/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.18.4
[pip3] numpydoc==1.0.0
[pip3] torch==1.5.0
[pip3] torchvision==0.2.2.post3
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] mkl                       2020.1                      217  
[conda] mkl-service               2.3.0            py37he904b0f_0  
[conda] mkl_fft                   1.0.15           py37ha843d7b_0  
[conda] mkl_random                1.1.1            py37h0573a6f_0  
[conda] numpy                     1.18.1           py37h4f9e942_0  
[conda] numpy-base                1.18.1           py37hde5b4d6_1  
[conda] pytorch                   1.5.0           py3.7_cuda10.2.89_cudnn7.6.5_0    pytorch
[conda] torchvision               0.6.0                py37_cu102    pytorch

Additional context

I managed to work around the issue by removing third_party/*, adding LIBS=-ltinfo at line 80
of build_tools/setup_helpers/build_third_party_helper.sh as such:

            ./configure ${CONFIG_OPTS} \
                        --disable-shared --enable-static --prefix="${install_dir}" CFLAGS=-fPIC CXXFLAGS=-fPIC \
                        LIBS=-ltinfo --with-pic --disable-debug --disable-dependency-tracking --enable-nasm 

and run python setup.py install again.

[mthrok]

Hi @mmxgn

Thanks for reporting. Seems like this is specific to Arch Linux, on which we do not regularly test torchaudio.
Since the fix is very small, would you like to open a PR?
If this change does not break the rest of the build, we can incorporate it.

[mmxgn]

Hi,

Sure, I will do that. I was unsure whether it was specific to Arch Linux. I currently ran on another issue with SoX not finding the locally-built flac. I will try to fix this as well and will open a new issue if that's ok.

[mthrok]

Hi,

Sure, I will do that. I was unsure whether it was specific to Arch Linux. I currently ran on another issue with SoX not finding the locally-built flac. I will try to fix this as well and will open a new issue if that's ok.

Yes, we would love hear the fix too.

BTW if the installation was working previously, by adding .use_external_sox at the root directory of the repo and depend on the externally managed sox too. I recently flipped the behavior to build own codecs by default. If this causes too much trouble, I will also consider reverting to the original default behavior.

[mmxgn]

Hello,
Adding .use_external_sox at the root directory of the repo fixes that issue for me. Should I open a new issue and comment that there?

[mmxgn]

Also, copying another workaround here for #666:

Apparently the problem is when using the bundled anaconda ncurses and is fixed when using ncurses 6-1 from conda-forge with:

conda install -c conda-forge ncurses

In this case, the fix I posted is not needed.

I don't know if that is specific to Arch Linux or not but I am happy to continue working on it with this manual fix. Might I suggest getting it into a FAQ or TROUBLESHOOTING subsection though in case it helps with others? Could also help if the workaround for sox is also included.

[mmxgn]

Hi,

I am closing this since the workaround worked and README.md has been updated :)

[prashantserai]

I am facing the same issue but the workaround(s) don't help. I tried installing ncurses with the provided conda install command as well as the hacky approach:
-trying to delete third_party/*
-there doesn't exist a file anymore that looks like build_tools/setup_helpers/build_third_party_helper.sh , so I couldn't hack it as mentioned above.

Plus, deleting third_party/* makes it complain that third_party/CMakeLists.txt is absent.