Static build issue with soxr

[joerg-krause]

The Buildroot autobuilder detected a build issue when building shairport-sync statically:

arm-buildroot-linux-uclibcgnueabi/sysroot/usr//lib/libsoxr.a(soxr.c.o): In function `soxr_create':
soxr.c:(.text+0xd4c): undefined reference to `av_get_cpu_flags'
collect2: error: ld returned 1 exit status

A quick look at the soxr source code shows that soxr_create() might use av_get_cpu_flags() under certain circumstances.

Therefore, -lavutil should be added to [other-libraries]. Even better, as soxr provides a pkg-config file, this should be prefered.

[mikebrady]

Thanks Jörg Krause. It looks great -- let me do a quick check on it today and hopefully accept the pull request.

[mikebrady]

Hi Jörg Krause. That avutil library is absolutely huge -- 186 MB of a download on a Raspberry Pi! I can't help thinking it's a mistake or oversight. What do you think?

[joerg-krause]

Did you mean KB?

[mikebrady]

Nope -- I really mean megabytes! I'm redoing everything again -- it'll probably take another day or so, and I'll record the log this time.

[mikebrady]

This is a transcript for looking for and installing labavutil-dev, which I think is the closest thing:

root@raspberrypi:/home/pi# apt-cache search avutil
libavutil-dev - FFmpeg library with functions for simplifying programming - development files
libavutil55 - FFmpeg library with functions for simplifying programming - runtime files
libavutil55-dbgsym - Debug symbols for libavutil55
root@raspberrypi:/home/pi# apt-get install libavutil-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libavutil55 libdrm-amdgpu1 libdrm-freedreno1 libdrm-nouveau2 libdrm-radeon1 libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa libllvm3.9
  libsensors4 libtxc-dxtn-s2tc libva-drm1 libva-x11-1 libva1 libvdpau-va-gl1 libvdpau1 libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0
  libxcb-present0 libxcb-sync1 libxdamage1 libxfixes3 libxshmfence1 libxxf86vm1 mesa-va-drivers mesa-vdpau-drivers va-driver-all
  vdpau-driver-all
Suggested packages:
  lm-sensors
The following NEW packages will be installed:
  libavutil-dev libavutil55 libdrm-amdgpu1 libdrm-freedreno1 libdrm-nouveau2 libdrm-radeon1 libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa
  libllvm3.9 libsensors4 libtxc-dxtn-s2tc libva-drm1 libva-x11-1 libva1 libvdpau-va-gl1 libvdpau1 libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0
  libxcb-present0 libxcb-sync1 libxdamage1 libxfixes3 libxshmfence1 libxxf86vm1 mesa-va-drivers mesa-vdpau-drivers va-driver-all
  vdpau-driver-all
0 upgraded, 30 newly installed, 0 to remove and 0 not upgraded.
Need to get 18.0 MB of archives.
After this operation, 185 MB of additional disk space will be used.
Do you want to continue? [Y/n] 

[mikebrady]

Apologies -- just (!) 18 MB of a download, but 185 MB of dis space needed!

[joerg-krause]

libavutil-dev itself is only about 343 kB on Debian Stretch, 1179 kB installed. This is probably a full-blown development build with all possible dependencies of FFmpeg.

[mikebrady]

Perhaps. The puzzling thing is that libavutil is not actually necessary for the packaged-up version of libsoxr to run in Raspbian or on Ubuntu, as far as I can tell. As an experiment, looking at the compilation of it on a fresh version of Raspbian Stretch, it looks like the cmake utility checks for the existence of libavutil and will include references to it if it's there. But if it's not there, that's fine:

pi@raspberrypi:~ $ git clone https://git.code.sf.net/p/soxr/code soxr-code
Cloning into 'soxr-code'...
remote: Enumerating objects: 1007, done.
remote: Counting objects: 100% (1007/1007), done.
remote: Compressing objects: 100% (899/899), done.
remote: Total 1007 (delta 684), reused 153 (delta 88)
Receiving objects: 100% (1007/1007), 293.07 KiB | 0 bytes/s, done.
Resolving deltas: 100% (684/684), done.
pi@raspberrypi:~ $ cd soxr-code
pi@raspberrypi:~/soxr-code $ ./go

-- The C compiler identification is GNU 6.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for pow in m
-- Looking for pow in m - found
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: -fopenmp  
-- Trying SIMD32 C flags: -mfpu=neon-vfpv4 -mcpu=cortex-a7
-- Performing Test DETECT_SIMD32_C_FLAGS
-- Performing Test DETECT_SIMD32_C_FLAGS - Success
-- Found SIMD32: -mfpu=neon-vfpv4 -mcpu=cortex-a7  
-- Could NOT find SIMD64 (missing:  SIMD64_C_FLAGS SIMD64_C_FLAGS) 
-- Could NOT find AVUTIL (missing:  AVUTIL_LIBRARIES AVUTIL_INCLUDE_DIRS) 
-- Looking for lrint
-- Looking for lrint - found
-- Looking for include file fenv.h
-- Looking for include file fenv.h - found
-- Looking for include file stdbool.h
-- Looking for include file stdbool.h - found
-- Looking for include file stdint.h
-- Looking for include file stdint.h - found
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Using unsigned short
-- Check if the system is big endian - little endian
-- Configuring done
-- Generating done
-- Build files have been written to: /home/pi/soxr-code/Release

By the way, this process seems to install version 0.1.2:

Install the project...
...
-- Installing: /usr/local/lib/libsoxr.so.0.1.2
...

At a guess, in the buildroot environment, is it possible that libavutil has been built and made available for cmake to find when it's configuring the build of libsoxr? If so, maybe building libsoxr before libavutil would be worth thinking about.

[joerg-krause]

At a guess, in the buildroot environment, is it possible that libavutil has been built and made available for cmake to find when it's configuring the build of libsoxr?

Correct.

If so, maybe building libsoxr before libavutil would be worth thinking about.

No. If libavutil is available, libsoxr should use it. Otherwise, libsoxr is not able to use some optimizations for certain platforms, like ARM NEON.

There is nothing wrong with libsoxr using libavutil. The only thing is that under certain circumstances libsoxr uses libavutil for the sake of optimization. Therefore, when checking for soxr_create, libavutil should be taken into account when using AC_CHECK_LIB.

[mikebrady]

Thanks.

The situation, therefore, is that libsoxr may depend on libavutil if libsoxr been compiled that way.

In Ubuntu and in Raspbian and (AFAIK) in OpenWrt, the version of libsoxr that is in the libraries is compiled without the libavutil dependency.

Therefore libavutil is not a dependency in those cases, and therefore, IMHO, it would be incorrect to make it a requirement when using AC_CHECK_LIB.

Now, if there was a way to check if libavutil was present in the build environment, and if it was, to make it a dependency, that would solve the problem (on the assumption that libsoxr was built with it as a requirement). It seems like it might be tricky!

[mikebrady]

I'm wondering if something like this might work:

# Look for soxr flag
AC_ARG_WITH(soxr, [  --with-soxr = choose libsoxr for high-quality interpolation], [
  AC_MSG_RESULT(>>Including support for soxr-based interpolation)
  if  test "x${with_pkg_config}" = xyes ; then
    PKG_CHECK_MODULES(
        [SOXR], [soxr],
        [LIBS="${SOXR_LIBS} ${LIBS}"])
  else
    AC_CHECK_LIB([avutil],[av_get_cpu_flags], ,AC_MSG_RESULT(Note: libavutil not found -- so not including it for libsoxr))
    AC_CHECK_LIB([soxr],[soxr_create], , AC_MSG_ERROR(soxr support requested but libsoxr not found!), )
  fi
], )

[joerg-krause]

Therefore libavutil is not a dependency in those cases, and therefore, IMHO, it would be incorrect to make it a requirement when using AC_CHECK_LIB.

I wasn't aware that AC_CHECK_LIB fails if libavutil is not found.

So, following your suggestion for the non-pkg-config branch I would suggest:

  if  test "x${with_pkg_config}" = xyes ; then
        ...
  else
	AC_CHECK_LIB([avutil],[av_get_cpu_flags])
	if  test "x${ac_cv_lib_avutil_av_get_cpu_flags}" = xyes ; then
                # soxr may link against libavutil, depending on the architecture, but for the sake of simplicity link with it if it is found
		AC_CHECK_LIB([soxr],[soxr_create], , AC_MSG_ERROR(soxr support requested but libsoxr not found!), [-lavutil])
	else
		AC_CHECK_LIB([soxr],[soxr_create], , AC_MSG_ERROR(soxr support requested but libsoxr not found!))
	fi
  fi

[mikebrady]

That looks nice, thanks. Let me check it out. Does it do the same thing as my suggestion?

[Update] I see that it's a bit more sophisticated. Let me check it out. Many thanks.

[mikebrady]

Is this what you had in mind?

If so, would you be kind enough to try it out and let us know how it goes. From my point of view, it seems okay -- it doesn't trigger unnecessary loading or linking of the libavutil library.

It is all in a (temporary!) new branch called avutilsoxr.

[joerg-krause]

Yes, this is exactly what I had in mind. Thanks!

[joerg-krause]

Closed in 5101ab2.