Non-system header builds (e.g. conda) have inconsistent /usr/include header checks

[taranu]

I ran into a few traps trying to build xpra v4.0+ with conda, primarily for a headless Centos 7 server that only had v3.1 available. Essentially, any of the checks pointed at /usr/include (e.g. in setup.py) aren't consistent with builds in other environments. For example, if you have a conda environment without kernel-headers_linux-64, linux/videodev2.h and linux/vm_sockets.h will be missing entirely (from e.g. $CONDA_PREFIX/x86_64-conda-linux-gnu/sysroot/usr/include/), and sysroot_linux-64<2.17 don't define v4l2_capability.device_caps, all of which will cause compilation errors if they differ from their /usr/include counterparts.

I'm not sure if there's any easy way to determine the actual include path of those headers. Having said that, with reference to providing a conda package, I have been able to build xpra up to 4.2.3 on python 3.9 as follows on both Centos and Fedora:

conda create -n xpra python=3.9
conda activate xpra
conda config --add channels conda-forge
# gcc brings kernel-headers_linux-64, necessary for:
# /usr/include/linux/videodev2.h and /usr/include/linux/vm_sockets.h
conda install cython gcc pkg-config pkgconfig xorg-libx11
conda install xorg-libxrandr xorg-libxtst xorg-libxfixes xorg-libxdamage xorg-libxcomposite xorg-util-macros
conda install xorg-libxcursor xorg-libxinerama xorg-xineramaproto xorg-libxscrnsaver
# gtk3 will not import without xorg-xineramaproto
conda install gtk3 pycairo pygobject
conda install x264 libvpx ffmpeg
conda install libxkbfile
# Docs will fail to build before python lib is installed otherwise
conda install pandoc
# defines v4l2_capability.device_caps
conda install sysroot_linux-64=2.17

git clone https://github.com/Xpra-org/xpra
git reset --hard tags/v4.2.3
python setup.py install

The current master branch appears to also require libxres, which I haven't found on conda; it can be built from source with the addition of xorg-xorgproto and either system or conda-provided aclocal, autoconf and automake.

[totaam]

all of which will cause compilation errors if they differ from their /usr/include counterparts

The commit above should make it easier for you to use a different include directory:

INCLUDE_DIR=$CONDA_PREFIX/x86_64-conda-linux-gnu/sysroot/usr/include/ ./setup.py build

(xpra) $ touch xpra/codecs/v4l2/pusher.pyx 
(xpra) $ python3 ./setup.py build
(..)
gcc -pthread -B /home/antoine/.conda/envs/xpra/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/antoine/.conda/envs/xpra/include -fPIC -O2 -isystem /home/antoine/.conda/envs/xpra/include -fPIC -Ixpra/codecs/v4l2 -I/home/antoine/.conda/envs/xpra/include/python3.10 -c xpra/codecs/v4l2/pusher.c -o build/temp.linux-x86_64-3.10/xpra/codecs/v4l2/pusher.o -Wall -Werror -fPIC
gcc -pthread -B /home/antoine/.conda/envs/xpra/compiler_compat -shared -Wl,-rpath,/home/antoine/.conda/envs/xpra/lib -Wl,-rpath-link,/home/antoine/.conda/envs/xpra/lib -L/home/antoine/.conda/envs/xpra/lib -Wl,-rpath,/home/antoine/.conda/envs/xpra/lib -Wl,-rpath-link,/home/antoine/.conda/envs/xpra/lib -L/home/antoine/.conda/envs/xpra/lib build/temp.linux-x86_64-3.10/xpra/codecs/v4l2/pusher.o -o build/lib.linux-x86_64-3.10/xpra/codecs/v4l2/pusher.cpython-310-x86_64-linux-gnu.so -Wall

Now running this again using INCLUDE_DIR=/home/antoine/.conda/envs/xpra/include/ python3 ./setup.py build makes no difference to the gcc command lines.
What am I missing?

Note: you're also missing a bunch of printing related dependencies, etc. I assume you're aware of that.

[taranu]

all of which will cause compilation errors if they differ from their /usr/include counterparts

The commit above should make it easier for you to use a different include directory:

INCLUDE_DIR=$CONDA_PREFIX/x86_64-conda-linux-gnu/sysroot/usr/include/ ./setup.py build

Thank you! That seemed to do the trick.

(xpra) $ touch xpra/codecs/v4l2/pusher.pyx 
(xpra) $ python3 ./setup.py build
(..)
gcc -pthread -B /home/antoine/.conda/envs/xpra/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/antoine/.conda/envs/xpra/include -fPIC -O2 -isystem /home/antoine/.conda/envs/xpra/include -fPIC -Ixpra/codecs/v4l2 -I/home/antoine/.conda/envs/xpra/include/python3.10 -c xpra/codecs/v4l2/pusher.c -o build/temp.linux-x86_64-3.10/xpra/codecs/v4l2/pusher.o -Wall -Werror -fPIC
gcc -pthread -B /home/antoine/.conda/envs/xpra/compiler_compat -shared -Wl,-rpath,/home/antoine/.conda/envs/xpra/lib -Wl,-rpath-link,/home/antoine/.conda/envs/xpra/lib -L/home/antoine/.conda/envs/xpra/lib -Wl,-rpath,/home/antoine/.conda/envs/xpra/lib -Wl,-rpath-link,/home/antoine/.conda/envs/xpra/lib -L/home/antoine/.conda/envs/xpra/lib build/temp.linux-x86_64-3.10/xpra/codecs/v4l2/pusher.o -o build/lib.linux-x86_64-3.10/xpra/codecs/v4l2/pusher.cpython-310-x86_64-linux-gnu.so -Wall

Now running this again using INCLUDE_DIR=/home/antoine/.conda/envs/xpra/include/ python3 ./setup.py build makes no difference to the gcc command lines. What am I missing?

If I were to omit the sysroot upgrade step I posted (as I did the first time I tried), I end up with sysroot_linux-64 2.12, and the following compilation error if I try to build without that commit (I reset to the previous commit 25ec8930eee0385ca21545f8c8e67214dbd90c78):

creating build/temp.linux-x86_64-3.9/xpra/codecs/v4l2
gcc -pthread -B /home/taranu/.conda/envs/xpra2/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/taranu/.conda/envs/xpra2/include -fPIC -O2 -isystem /home/taranu/.conda/envs/xpra2/include -fPIC -Ixpra/codecs/v4l2 -I/home/taranu/.conda/envs/xpra2/include/python3.9 -c xpra/codecs/v4l2/pusher.c -o build/temp.linux-x86_64-3.9/xpra/codecs/v4l2/pusher.o -Wall -Werror -fPIC
xpra/codecs/v4l2/pusher.c: In function '__pyx_pf_4xpra_6codecs_4v4l2_6pusher_query_video_device':
xpra/codecs/v4l2/pusher.c:2951:74: error: 'struct v4l2_capability' has no member named 'device_caps'
 2951 |                   __pyx_t_14 = __Pyx_PyInt_From_uint32_t(__pyx_v_vid_caps.device_caps); if (unlikely(!__pyx_t_14)) __PYX_ERR(0, 267, __pyx_L29_error)
      |                                                                          ^
error: command '/home/taranu/.conda/envs/xpra2/bin/gcc' failed with exit code 1

(Because my /usr/include/linux/videodev2.h does have device_caps).

After upgrading sysroot to 2.17 and after install libxres, and reset to master, it either fails to find $CONDA_PREFIX/include/linux/videodev2.h without INCLUDE_DIR specified, or compiles correctly via INCLUDE_DIR=$CONDA_PREFIX/x86_64-conda-linux-gnu/sysroot/usr/include ./setup.py build with all references to device_caps in xpra/codecs/v4l2/pusher.c commented out. Yes, the compile commands are the same - I think the only difference is that xpra/codecs/v4l2/constants.pxi has DEF ENABLE_DEVICE_CAPS=0 via setup.py.

Note: you're also missing a bunch of printing related dependencies, etc. I assume you're aware of that.

I wasn't, but thanks. I have to admit I didn't carefully check that all dependencies were installed but just added the bare minimum to compile successfully.

[totaam]

Yes, the compile commands are the same - I think the only difference is that xpra/codecs/v4l2/constants.pxi has DEF ENABLE_DEVICE_CAPS=0 via setup.py

It should have already used the correct file for the feature detection.
The commit above ensures that gcc will pick the correct file first when using INCLUDE_DIR and also more correctly check for device_caps.
Tested by commenting out device_caps and copying the header file to /usr/local/include/linux/video/ then compiling with:

INCLUDE_DIR=/usr/local/include/ ./setup.py build

Does that work for you?

[taranu]

Sorry, perhaps I wasn't clear. I just meant that the linux headers provided by conda's sysroot package don't get installed to $CONDA_PREFIX/include/linux but $CONDA_PREFIX/x86_64-conda-linux-gnu/sysroot/usr/include, so you have to specify the latter as INCLUDE_DIRto use videodev2at all.

Put another way, if there's only one INCLUDE_DIRin setup.py you can't mix and match features from conda vs system packages, correct? For example, I don't see pam_misc.h in either of those conda include paths (Fedora 35 has it in /usr/include/security/pam_misc.h), so if I specify INCLUDE_DIR then it won't enable PAM support, correct?

[totaam]

Ah, gotcha. The sysroot is incomplete so you need both dirs.
The commit above should do what you want. Briefly tested with:

INCLUDE_DIRS=/usr/include/:/usr/local/include/ python3 ./setup.py build

[totaam]

My guess is that INCLUDE_DIRS is not enough and you will also need LIB_DIRS..

[taranu]

Thanks. I tried LIBRARY_PATH=/usr/lib:/usr/lib64 INCLUDE_DIRS=/usr/include:$CONDA_PREFIX/x86_64-conda-linux-gnu/sysroot/usr/include:$CONDA_PREFIX/usr/include ./setup.py install and that worked to enable PAM support. Note that I can't use /usr/local/include on one of the machines I test on since I don't have root access.

[totaam]

OK, so I can finally close this ticket.

Note that I can't use /usr/local/include

That was just an example.