npm install fails with compilation error

[lelikg]

Operating system:

Debian Buster

• Node version:
  18.14
• npm version:
  9.3.1
• mediasoup version:
  latest

Issue description

When running npm install, fails with:

#12 177.3 npm ERR! [778/1349] Compiling C++ object mediasoup-worker.p/src_RTC_Transport.cpp.o
#12 177.3 npm ERR! FAILED: mediasoup-worker.p/src_RTC_Transport.cpp.o 
#12 177.3 npm ERR! c++ -Imediasoup-worker.p -I. -I../../.. -I../../../include -Isubprojects/abseil-cpp-20230802.0 -I../../../subprojects/abseil-cpp-20230802.0 -I../../../subprojects/openssl-3.0.8/include -I../../../subprojects/openssl-3.0.8/crypto -I../../../subprojects/openssl-3.0.8/crypto/modes -I../../../subprojects/openssl-3.0.8/crypto/ec/curve448 -I../../../subprojects/openssl-3.0.8/crypto/ec/curve448/arch_32 -I../../../subprojects/openssl-3.0.8/providers/common/include -I../../../subprojects/openssl-3.0.8/providers/implementations/include -Isubprojects/openssl-3.0.8/generated-config/archs/linux-x86_64/asm -I../../../subprojects/openssl-3.0.8/generated-config/archs/linux-x86_64/asm -I../../../subprojects/openssl-3.0.8/generated-config/archs/linux-x86_64/asm/include -I../../../subprojects/openssl-3.0.8/generated-config/archs/linux-x86_64/asm/crypto -I../../../subprojects/openssl-3.0.8/generated-config/archs/linux-x86_64/asm/providers/common/include -I../../../subprojects/libuv-v1.47.0/include -Isubprojects/libsrtp-2.5.0/include -I../../../subprojects/libsrtp-2.5.0/include -Isubprojects/usrsctp-ebb18adac6501bad4501b1f6dccb67a1c85cc299/usrsctplib -I../../../subprojects/usrsctp-ebb18adac6501bad4501b1f6dccb67a1c85cc299/usrsctplib -Isubprojects/usrsctp-ebb18adac6501bad4501b1f6dccb67a1c85cc299/usrsctplib/netinet -I../../../subprojects/usrsctp-ebb18adac6501bad4501b1f6dccb67a1c85cc299/usrsctplib/netinet -Isubprojects/usrsctp-ebb18adac6501bad4501b1f6dccb67a1c85cc299/usrsctplib/netinet6 -I../../../subprojects/usrsctp-ebb18adac6501bad4501b1f6dccb67a1c85cc299/usrsctplib/netinet6 -I../../../subprojects/flatbuffers-23.3.3/include -I../../../subprojects/flatbuffers-23.3.3/grpc -Ifbs -I../../../fbs -Ideps/libwebrtc -I../../../deps/libwebrtc -I../../../deps/libwebrtc/libwebrtc -Isubprojects/liburing-liburing-2.4/src/include -I../../../subprojects/liburing-liburing-2.4/src/include -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++17 -O3 -fPIE -pthread -DMS_LITTLE_ENDIAN -DMS_LIBURING_SUPPORTED -DMS_EXECUTABLE -MD -MQ mediasoup-worker.p/src_RTC_Transport.cpp.o -MF mediasoup-worker.p/src_RTC_Transport.cpp.o.d -o mediasoup-worker.p/src_RTC_Transport.cpp.o -c ../../../src/RTC/Transport.cpp
#12 177.3 npm ERR! In file included from ../../../subprojects/liburing-liburing-2.4/src/include/liburing.h:24,
#12 177.3 npm ERR!                  from ../../../include/DepLibUring.hpp:7,
#12 177.3 npm ERR!                  from ../../../src/RTC/Transport.cpp:7:
#12 177.3 npm ERR! subprojects/liburing-liburing-2.4/src/include/liburing/compat.h:5:10: fatal error: linux/time_types.h: No such file or directory
#12 177.3 npm ERR!  #include <linux/time_types.h>
#12 177.3 npm ERR!           ^~~~~~~~~~~~~~~~~~~~
#12 177.3 npm ERR! compilation terminated.
#12 177.3 npm ERR! [779/1349] Compiling C++ object mediasoup-worker.p/src_RTC_TcpServer.cpp.o

[ibc]

The configure script of liburing defines UAPI_LINUX_IO_URING_H_SKIP_LINUX_TIME_TYPES_H macro which prevents the offending linux/time_types.h file from being included:

https://github.com/axboe/liburing/blob/63f0974a5f4bbc8f18abebe63ac630fe82f77e52/configure#L536C1-L536C1

Probably Meson wrap file doesn't do it.

[jmillan]

I've already tested this behaviour by removing time_types.h from the system and it failed to me once, but I could not reproduce. The rest of the times it compiled correctly by replacing the time definitions.

[ibc]

I've already tested this behaviour by removing time_types.h from the system and it failed to me once, but I could not reproduce. The rest of the times it compiled correctly by replacing the time definitions.

I don't understand what you mean. I just said that the official configure script of liburing does checks to see if it should define that macro or not, while the meson wrap file doesn't.

This is the meson.build file of liburing. It doesn't setup the UAPI_LINUX_IO_URING_H_SKIP_LINUX_TIME_TYPES_H macro at all and it doesn't check the existence of linux/time_types.h at all:

project('liburing', ['c','cpp'],
        version: '2.4',
        license: ['MIT', 'LGPL-2.1-only', 'GPL-2.0-only WITH Linux-syscall-note'],
        meson_version: '>=0.57.0',
        default_options: ['default_library=both',
                          'buildtype=debugoptimized',
                          'c_std=c11',
                          'cpp_std=c++11',
                          'warning_level=2'])

if host_machine.system() != 'linux'
  error('Everything except Linux is unsupported. uring is a Linux framework')
endif

project_version = meson.project_version().split('.')
major_version = project_version[0]
minor_version = project_version[1]

add_project_arguments('-D_GNU_SOURCE',
                      '-D__SANE_USERSPACE_TYPES__',
                      '-include', meson.current_build_dir() + '/config-host.h',
                      '-fno-stack-protector',
                      '-Wno-unused-parameter',
                      language: ['c', 'cpp'])

thread_dep = dependency('threads')

cc = meson.get_compiler('c')

has_stringop_overflow = cc.has_argument('-Wstringop-overflow')
has_array_bounds = cc.has_argument('-Warray-bounds')

has__kernel_rwf_t = cc.has_type('__kernel_rwf_t', prefix: '#include <linux/fs.h>')

has__kernel_timespec = cc.has_members('struct __kernel_timespec',
                                      'tv_sec',
                                      'tv_nsec',
                                      prefix: '#include <linux/time.h>')

code = '''#include <sys/types.h>
#include <fcntl.h>
#include <string.h>
#include <linux/openat2.h>
int main(int argc, char **argv)
{
  struct open_how how;
  how.flags = 0;
  how.mode = 0;
  how.resolve = 0;
  return 0;
}
'''
has_open_how = cc.compiles(code, name: 'open_how')

code = '''#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#include <linux/stat.h>
int main(int argc, char **argv)
{
  struct statx x;

  return memset(&x, 0, sizeof(x)) != NULL;
}
'''
has_statx = cc.compiles(code, name: 'statx')

code= '''#include <sys/types.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#include <linux/stat.h>
main(int argc, char **argv)
{
  struct statx x;

  return memset(&x, 0, sizeof(x)) != NULL;
}
'''
glibc_statx = cc.compiles(code, name: 'glibc_statx')

cpp = meson.get_compiler('cpp')

code = '''#include <iostream>
int main(int argc, char **argv)
{
  std::cout << "Test";
  return 0;
}
'''
has_cxx = cpp.compiles(code, name: 'C++')

has_ucontext = (cc.has_type('ucontext_t', prefix: '#include <ucontext.h>')
  and cc.has_function('makecontext', prefix: '#include <ucontext.h>'))

has_nvme_uring_cmd = cc.has_type('struct nvme_uring_cmd', prefix: '#include <linux/nvme_ioctl.h>')

conf_data = configuration_data()
conf_data.set('CONFIG_HAVE_KERNEL_RWF_T', has__kernel_rwf_t)
conf_data.set('CONFIG_HAVE_KERNEL_TIMESPEC', has__kernel_timespec)
conf_data.set('CONFIG_HAVE_OPEN_HOW', has_open_how)
conf_data.set('CONFIG_HAVE_STATX', has_statx)
conf_data.set('CONFIG_HAVE_GLIBC_STATX', glibc_statx)
conf_data.set('CONFIG_HAVE_CXX', has_cxx)
conf_data.set('CONFIG_HAVE_UCONTEXT', has_ucontext)
conf_data.set('CONFIG_HAVE_NVME_URING', has_nvme_uring_cmd)
conf_data.set('CONFIG_HAVE_FANOTIFY', cc.has_header('sys/fanotify.h'))
configure_file(output: 'config-host.h',
               configuration: conf_data)

subdir('src')
subdir('man')

if get_option('examples')
    subdir('examples')
endif

if get_option('tests')
    if get_option('default_library') != 'both'
        error('default_library=both required to build tests')
    endif
    subdir('test')
endif

pkg_mod = import('pkgconfig')
pkg_mod.generate(libraries: liburing,
                 name: 'liburing',
                 version: meson.project_version(),
                 description: 'io_uring library',
                 url: 'http://git.kernel.dk/cgit/liburing/')

summary({'bindir': get_option('bindir'),
         'libdir': get_option('libdir'),
         'datadir': get_option('datadir'),
        }, section: 'Directories')
summary({'examples': get_option('examples'),
         'tests': get_option('tests')
        }, section: 'Configuration', bool_yn: true)

[jmillan]

@ibc, there are many meson.build files in liburing. This one exactly is the one building the compat.h file which is the file created at build time and the one that includes or not the time_types.h based on its existence.

https://github.com/mesonbuild/wrapdb/tree/master/subprojects/packagefiles/liburing/src/include/liburing

@lelikg, we need the full logs for the installation.

I can only reproduce the problem if I follow these steps:

1. time_types.h is in my system.
2. worker$ make -> meson setup (this will notice that time_types.h exists and include it in compat.h)
3. remove time_types.h from the system
4. meson setup --reconfigure (this won't check that time_types.h is not present and will not update compat.h)
5. Since compat.h does include a file that does not exists, the error that is happening will occur.

Which in a npm install process doesn't seem to be a feasible scenario.

[jmillan]

@lelikg,

Is it possible that you are running npm install, then some system header files change, and later you are calling npm install again?

This is a feasible (but buggy) scenario when creating Docker images where you create the images in different steps.

[lelikg]

@jmillan I'll try to get the full logs for you. meanwhile I can say that I set the mediasoup version to "3.13.6" and the build works

[lelikg]

@jmillan here is a complete log of the docker build scratch_63.txt

[jmillan]

@lelikg, I don't see the very exact problem here, but you are using Debian Buster (oldoldstable) which comes with the kernel version 4.19.

We only enable liburing for kernel versions >=6. This means that your Docker/Kubernetes executable has a kernel version >=6, but you are creating an image with an old Debian. The image uses Docker/Kubernets kernel, this is why it's being enabled in this case.

I would suggest you to upgrade to at least the current stable Debian version (bookworm), for many reasons, and try to install mediasoup version 3.3.17 again.

[ibc]

Probably irrelevant, but the prebuilt is failing so local build is done. This is a known issue with some not super recent Debian versions AFAIR:

npm-scripts.mjs �[36m[INFO] [postinstall]�[0m downloadPrebuiltWorker() | got mediasoup-worker prebuilt binary
2023-12-04T17:08:36.0738600Z #12 177.3 npm ERR! npm-scripts.mjs �[36m[INFO] [postinstall]�[0m downloadPrebuiltWorker() | checking fetched mediasoup-worker prebuilt binary in current host
2023-12-04T17:08:36.0740299Z #12 177.3 npm ERR! npm-scripts.mjs �[36m[INFO] [postinstall]�[0m couldn't fetch any mediasoup-worker prebuilt binary, building it locally

[jmillan]

@lelikg,

Can you please run npm install --foreground-scripts and attach the logs. They will hopefully show how liburing objects are compiled.

[jmillan]

@lelikg,

Please add the package linux-libc-dev to the apt-get install directive before running npm install. It should make it as it will install linux/time_types.h header file.

We're working in fixing the root problem.

[jmillan]

Ongoing fix in wrapdb

[lelikg]

@lelikg,

Please add the package linux-libc-dev to the apt-get install directive before running npm install. It should make it as it will install linux/time_types.h header file.

We're working in fixing the root problem.

Hi @jmillan thanks a lot, I added what you suggested and it works.
Great job on the library it's amazing

Leon