Can I disable the use of statx() syscall even though it's available on the build system?

[Gei0r]

During build, the test has_statx_syscall in Jamfile.v2 is perfomed to check if statx() is available. Because it is available on my build system, boost::filesystem uses the statx syscall.
However, my executable also has to run on older kernels which don't have this syscall yet (before 4.11).

How can I disable the use of the statx syscall, preferably in the b2 invocation while building boost::filesystem?

[Lastique]

There is no switch to disable statx, or any other modern system APIs used in Boost.Filesystem. This is deliberate. The recommendation is to build Boost on the system that is equivalent to that which will run the executable. That means both libc and kernel headers must match (or be older than) the kernel that will run the executable. Another solution is to patch your version of Boost, of course.

[luke-jr]

The recommendation is to build Boost on the system that is equivalent to that which will run the executable. That means both libc and kernel headers must match (or be older than) the kernel that will run the executable.

But it doesn't mean that. It's pretty common to have kernel headers newer than the actual kernel, even with only one system involved (ie, build system itself now can't run it!). Boost is the first project I've heard of having a problem with this.

[luke-jr]

Furthermore, it isn't actually possible to use old linux-headers anymore. glibc build fails with:

configure: error: GNU libc requires kernel header files from
Linux 3.2.0 or later to be installed before configuring.

[Lastique]

If you're using an old kernel then you might have to use older userspace, including libc and Boost.

[luke-jr]

That's unreasonable.

[Lastique]

Not at all. Newer software is expected to require newer third party components. Not that Boost.Filesystem requires - it doesn't, as long as you use proper headers. From my POV, supporting inconsistent configurations is what's unreasonable. The fact that this is to support Linux kernel 2.6.32, which was released 9 years ago or more, depending on the patch version, is not a compelling argument either.

Boost.Filesystem uses a number of modern Linux syscalls, and that number will probably grow in the future. Introducing an option for each of them is unmaintainable in the long term. And it will not resolve the issue in general anyway, because some functionality may not involve a separate syscall, but a macro or a parameter value for example. So no, mismatched configs are not supported, period.

[luke-jr]

From my POV, supporting inconsistent configurations is what's unreasonable.

It is a normal configuration to have a newer headers than kernel.

Boost.Filesystem uses a number of modern Linux syscalls, and that number will probably grow in the future. Introducing an option for each of them is unmaintainable in the long term.

I agree. Generally, well-designed software can simply fall back to other (often standards-defined) methods when the optimal code path is unsupported.

For example (just one instance, optimised for smallest diff size):

diff --git a/operations.cpp b/operations.cpp
index fc853fb..0397fe6 100644
--- a/operations.cpp
+++ b/operations.cpp
@@ -1202,25 +1202,33 @@ bool copy_file(const path& from, const path& to, unsigned int options, error_cod
     break;
   }
 
+  mode_t from_mode;
 #if defined(BOOST_FILESYSTEM_HAS_STATX) || defined(BOOST_FILESYSTEM_HAS_STATX_SYSCALL)
   unsigned int statx_data_mask = STATX_TYPE | STATX_MODE | STATX_INO | STATX_SIZE;
   if ((options & static_cast< unsigned int >(copy_options::update_existing)) != 0u)
     statx_data_mask |= STATX_MTIME;
 
+{
   struct ::statx from_stat;
   if (BOOST_UNLIKELY(statx(infile.fd, "", AT_EMPTY_PATH | AT_NO_AUTOMOUNT, statx_data_mask, &from_stat) < 0))
   {
-  fail_errno:
-    err = errno;
-    goto fail;
+    if (errno == ENOSYS)
+    {
+      goto fallback:
+    }
+    goto fail_errno;
   }
 
-  if (BOOST_UNLIKELY((from_stat.stx_mask & statx_data_mask) != statx_data_mask))
+  if (BOOST_LIKELY((from_stat.stx_mask & statx_data_mask) == statx_data_mask))
   {
-    err = ENOSYS;
-    goto fail;
+    from_mode = get_mode(from_stat);
+    goto have_mode;
   }
-#else
+}
+
+fallback:
+#endif
+{
   struct ::stat from_stat;
   if (BOOST_UNLIKELY(::fstat(infile.fd, &from_stat) != 0))
   {
@@ -1228,9 +1236,11 @@ bool copy_file(const path& from, const path& to, unsigned int options, error_cod
     err = errno;
     goto fail;
   }
-#endif
 
-  const mode_t from_mode = get_mode(from_stat);
+  from_mode = get_mode(from_stat);
+}
+
+have_mode:
   if (BOOST_UNLIKELY(!S_ISREG(from_mode)))
   {
     err = ENOSYS;

[Lastique]

It is a normal configuration to have a newer headers than kernel.

I don't consider that normal. That's certainly not the case with the systems I work with.

[Gei0r]

I'm inclined to agree that mismatching headers and used kernel are not common. For example, with crosstool-NG, you can select which kernel headers to install to match the target system kernel.

However, I believe my original use case (build on a newer system than is used for running) is common. With regards to statx(), boost::filesystem already includes the fallback code, using regular old stat(). I believe making the fallback detection in the Jamfile configurable would not impact maintainability.

[luke-jr]

I'm inclined to agree that mismatching headers and used kernel are not common.

This is the standard behaviour for Gentoo... linux-headers is installed and updated independently from whatever kernel is being used (which users expect to be able to swap-and-go without needing changes to userspace, so building against the latest headers is needed to avoid a rebuild when you switch to a new kernel).

However, I believe my original use case (build on a newer system than is used for running) is common.

Indeed, that's another real-world issue that we're going to have to address in Bitcoin Core. We have some minimal glue to enable users to run with ancient glibcs (missing newer versions of standard functions), but didn't realise Boost has this problem. :/