systemd: 247.6 -> 249.4

This updates systemd to version v249.4 from version v247.6.

Besides the many new features that can be found in the upstream
repository they also introduced a bunch of cleanup which ended up
requiring a few more patches on our side.

a) 0022-core-Handle-lookup-paths-being-symlinks.patch:
  The way symlinked units were handled was changed in such that the last
  name of a unit file within one of the unit directories
  (/run/systemd/system, /etc/systemd/system, ...) is used as the name
  for the unit. Unfortunately that code didn't take into account that
  the unit directories themselves could already be symlinks and thus
  caused all our units to be recognized slightly different.

  There is an upstream PR for this new patch:
    systemd/systemd#20479

b) The way the APIVFS is setup has been changed in such a way that we
   now always have /run. This required a few changes to the
   confinement tests which did assert that they didn't exist. Instead of
   adding another patch we can just adopt the upstream behavior. An
   empty /run doesn't seem harmful.

   As part of this work I refactored the confinement test just a little
   bit to allow better debugging of test failures. Previously it would
   just fail at some point and it wasn't obvious which of the many
   commands failed or what the unexpected string was. This should now be
   more obvious.

c) Again related to the confinement tests the way a file was tested for
   being accessible was optimized. Previously systemd would in some
   situations open a file twice during that check. This was reduced to
   one operation but required the procfs to be mounted in a units
   namespace.

   An upstream bug was filed and fixed. We are now carrying the
   essential patch to fix that issue until it is backported to a new
   release (likely only version 250). The good part about this story is
   that upstream systemd now has a test case that looks very similar to
   one of our confinement tests. Hopefully that will lead to less
   friction in the long run.

   systemd/systemd#20514
   systemd/systemd#20515

d) Previously we could grep for dlopen( somewhat reliably but now
   upstream started using a wrapper around dlopen that is most of the
   time used with linebreaks. This makes using grep not ergonomic
   anymore.

   With this bump we are grepping for anything that looks like a
   dynamic library name (in contrast to a dlopen(3) call) and replace
   those instead. That seems more robust. Time will tell if this holds.

   I tried using coccinelle to patch all those call sites using its
   tooling but unfornately it does stumble upon the _cleanup_
   annotations that are very common in the systemd code.

e) We now have some machinery for libbpf support in our systemd build.
   That being said it doesn't actually work as generating some skeletons
   doesn't work just yet. It fails with the below error message and is
   disabled by default (in both minimal and the regular build).

   > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h
   > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h
   > libbpf: elf: socket_bind_bpf is not a valid eBPF object file
   > Error: failed to open BPF object file: BPF object format invalid
   > Traceback (most recent call last):
   >   File "/build/source/tools/build-bpf-skel.py", line 128, in <module>
   >     bpf_build(args)
   >   File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build
   >     gen_bpf_skeleton(bpftool_exec=args.bpftool_exec,
   >   File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton
   >     skel = subprocess.check_output(bpftool_args, universal_newlines=True)
   >   File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output
   >     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
   >   File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run
   >     raise CalledProcessError(retcode, process.args,
   > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255.
   > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut)
   > ninja: build stopped: subcommand failed.

  f) We do now have support for TPM2 based disk encryption in our
     systemd build. The actual bits and pieces to make use of that are
     missing but there are various ongoing efforts in that direction.
     There is also the story about systemd in our initrd to enable this
     being used for root volumes. None of this will yet work out of the
     box but we can start improving on that front.

  g) FIDO2 support was added systemd and consequently we can now use
     that. Just with TPM2 there hasn't been any integration work with
     NixOS and instead this just adds that capability to work on that.

Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>

nixos/doc/manual/from_md/release-notes/rl-2111.section.xml

@@ -37,6 +37,11 @@
           PostgreSQL now defaults to major version 13.
         </para>
       </listitem>
+      <listitem>
+        <para>
+          Systemd was updated to version 249 (from 247).
+        </para>
+      </listitem>
     </itemizedlist>
   </section>
   <section xml:id="sec-release-21.11-new-services">

nixos/doc/manual/release-notes/rl-2111.section.md

@@ -14,6 +14,8 @@ In addition to numerous new and upgraded packages, this release has the followin
 
 - PostgreSQL now defaults to major version 13.
 
+- Systemd was updated to version 249 (from 247).
+
 ## New Services {#sec-release-21.11-new-services}
 
 - [btrbk](https://digint.ch/btrbk/index.html), a backup tool for btrfs subvolumes, taking advantage of btrfs specific capabilities to create atomic snapshots and transfer them incrementally to your backup locations. Available as [services.btrbk](options.html#opt-services.brtbk.instances).

nixos/modules/services/ttys/getty.nix

@@ -131,6 +131,14 @@ in
         restartIfChanged = false;
       };
 
+    systemd.services."autovt@" =
+      { serviceConfig.ExecStart = [
+          "" # override upstream default with an empty ExecStart
+          (gettyCmd "--noclear %I $TERM")
+        ];
+        restartIfChanged = false;
+      };
+
     systemd.services."container-getty@" =
       { serviceConfig.ExecStart = [
           "" # override upstream default with an empty ExecStart

nixos/tests/systemd-confinement.nix

@@ -44,38 +44,35 @@ import ./make-test-python.nix {
       { config.confinement.mode = "chroot-only";
         testScript = ''
           with subtest("chroot-only confinement"):
-              machine.succeed(
-                  'test "$(chroot-exec ls -1 / | paste -sd,)" = bin,nix',
-                  'test "$(chroot-exec id -u)" = 0',
-                  "chroot-exec chown 65534 /bin",
-              )
+              paths = machine.succeed('chroot-exec ls -1 / | paste -sd,').strip()
+              assert_eq(paths, "bin,nix,run")
+              uid = machine.succeed('chroot-exec id -u').strip()
+              assert_eq(uid, "0")
+              machine.succeed("chroot-exec chown 65534 /bin")
         '';
       }
       { testScript = ''
           with subtest("full confinement with APIVFS"):
-              machine.fail(
-                  "chroot-exec ls -l /etc",
-                  "chroot-exec ls -l /run",
-                  "chroot-exec chown 65534 /bin",
-              )
-              machine.succeed(
-                  'test "$(chroot-exec id -u)" = 0',
-                  "chroot-exec chown 0 /bin",
-              )
+              machine.fail("chroot-exec ls -l /etc")
+              machine.fail("chroot-exec chown 65534 /bin")
+              assert_eq(machine.succeed('chroot-exec id -u').strip(), "0")
+              machine.succeed("chroot-exec chown 0 /bin")
         '';
       }
       { config.serviceConfig.BindReadOnlyPaths = [ "/etc" ];
         testScript = ''
           with subtest("check existence of bind-mounted /etc"):
-              machine.succeed('test -n "$(chroot-exec cat /etc/passwd)"')
+              passwd = machine.succeed('chroot-exec cat /etc/passwd').strip()
+              assert len(passwd) > 0, "/etc/passwd must not be empty"
         '';
       }
       { config.serviceConfig.User = "chroot-testuser";
         config.serviceConfig.Group = "chroot-testgroup";
         testScript = ''
           with subtest("check if User/Group really runs as non-root"):
               machine.succeed("chroot-exec ls -l /dev")
-              machine.succeed('test "$(chroot-exec id -u)" != 0')
+              uid = machine.succeed('chroot-exec id -u').strip()
+              assert uid != "0", "UID of chroot-testuser shouldn't be 0"
               machine.fail("chroot-exec touch /bin/test")
         '';
       }
@@ -88,10 +85,8 @@ import ./make-test-python.nix {
         testScript = ''
           with subtest("check if symlinks are properly bind-mounted"):
               machine.fail("chroot-exec test -e /etc")
-              machine.succeed(
-                  "chroot-exec cat ${symlink} >&2",
-                  'test "$(chroot-exec cat ${symlink})" = "got me"',
-              )
+              text = machine.succeed('chroot-exec cat ${symlink}').strip()
+              assert_eq(text, "got me")
         '';
       })
       { config.serviceConfig.User = "chroot-testuser";
@@ -158,6 +153,9 @@ import ./make-test-python.nix {
   };
 
   testScript = { nodes, ... }: ''
+    def assert_eq(a, b):
+        assert a == b, f"{a} != {b}"
+
     machine.wait_for_unit("multi-user.target")
   '' + nodes.machine.config.__testSteps;
 }

pkgs/development/libraries/tpm2-tss/default.nix

@@ -3,6 +3,14 @@
 , openssl, json_c, curl, libgcrypt
 , cmocka, uthash, ibm-sw-tpm2, iproute2, procps, which
 }:
+let
+  # Avoid a circular dependency on Linux systems (systemd depends on tpm2-tss,
+  # tpm2-tss tests depend on procps, procps depends on systemd by default). This
+  # needs to be conditional based on isLinux because procps for other systems
+  # might not support the withSystemd option.
+  procpsWithoutSystemd = procps.override { withSystemd = false; };
+  procps_pkg = if stdenv.isLinux then procpsWithoutSystemd else procps;
+in
 
 stdenv.mkDerivation rec {
   pname = "tpm2-tss";
@@ -20,7 +28,7 @@ stdenv.mkDerivation rec {
   ];
   buildInputs = [ openssl json_c curl libgcrypt ];
   checkInputs = [
-    cmocka uthash ibm-sw-tpm2 iproute2 procps which
+    cmocka uthash ibm-sw-tpm2 iproute2 procps_pkg which
   ];
 
   preAutoreconf = "./bootstrap";

pkgs/os-specific/linux/systemd/0001-Start-device-units-for-uninitialised-encrypted-devic.patch

@@ -1,4 +1,4 @@
-From 2f4a5e9c9ef1cd57662e8bd4c24e1029a00d55b5 Mon Sep 17 00:00:00 2001
+From 57e31a2d4a5d5bd7a9e1cd8a0d8bc6a00624ad68 Mon Sep 17 00:00:00 2001
 From: Eelco Dolstra <eelco.dolstra@logicblox.com>
 Date: Tue, 8 Jan 2013 15:46:30 +0100
 Subject: [PATCH 01/19] Start device units for uninitialised encrypted devices
@@ -13,7 +13,7 @@ unit.  (However, this ignores the fsck unit, so it's not perfect...)
  1 file changed, 4 deletions(-)
 
 diff --git a/rules.d/99-systemd.rules.in b/rules.d/99-systemd.rules.in
-index 7c22eefdb7..e3a55e00b5 100644
+index 25b8a590a6..d18999ea87 100644
 --- a/rules.d/99-systemd.rules.in
 +++ b/rules.d/99-systemd.rules.in
 @@ -17,10 +17,6 @@ SUBSYSTEM=="ubi", TAG+="systemd"
@@ -28,5 +28,5 @@ index 7c22eefdb7..e3a55e00b5 100644
  SUBSYSTEM=="block", ENV{ID_PART_GPT_AUTO_ROOT}=="1", ENV{ID_FS_TYPE}!="crypto_LUKS", SYMLINK+="gpt-auto-root"
  SUBSYSTEM=="block", ENV{ID_PART_GPT_AUTO_ROOT}=="1", ENV{ID_FS_TYPE}=="crypto_LUKS", SYMLINK+="gpt-auto-root-luks"
 -- 
-2.30.1
+2.32.0
 

pkgs/os-specific/linux/systemd/0002-Don-t-try-to-unmount-nix-or-nix-store.patch

@@ -1,4 +1,4 @@
-From 4e96b2e074c4a4f4ce900409872ce2f86704ee5b Mon Sep 17 00:00:00 2001
+From 43465a392b47238a32f8719f603ed9e2c9bb0363 Mon Sep 17 00:00:00 2001
 From: Eelco Dolstra <eelco.dolstra@logicblox.com>
 Date: Fri, 12 Apr 2013 13:16:57 +0200
 Subject: [PATCH 02/19] Don't try to unmount /nix or /nix/store
@@ -12,7 +12,7 @@ https://github.com/NixOS/nixos/issues/126
  2 files changed, 4 insertions(+)
 
 diff --git a/src/shared/fstab-util.c b/src/shared/fstab-util.c
-index 292b97cd69..791b8e6b7e 100644
+index f683f05981..5a04c2c2a6 100644
 --- a/src/shared/fstab-util.c
 +++ b/src/shared/fstab-util.c
 @@ -40,6 +40,8 @@ bool fstab_is_extrinsic(const char *mount, const char *opts) {
@@ -25,10 +25,10 @@ index 292b97cd69..791b8e6b7e 100644
                          "/etc"))
                  return true;
 diff --git a/src/shutdown/umount.c b/src/shutdown/umount.c
-index 3a72a13e1a..541320dc9d 100644
+index c2a26242c0..9936398f32 100644
 --- a/src/shutdown/umount.c
 +++ b/src/shutdown/umount.c
-@@ -500,6 +500,8 @@ static int delete_md(MountPoint *m) {
+@@ -496,6 +496,8 @@ static int delete_md(MountPoint *m) {
 
  static bool nonunmountable_path(const char *path) {
          return path_equal(path, "/")
@@ -38,5 +38,5 @@ index 3a72a13e1a..541320dc9d 100644
                  || path_equal(path, "/usr")
  #endif
 -- 
-2.30.1
+2.32.0
 

pkgs/os-specific/linux/systemd/0003-Fix-NixOS-containers.patch

@@ -1,4 +1,4 @@
-From 3d1b2e56a6ed6cc86a64f6f89765a2900e576402 Mon Sep 17 00:00:00 2001
+From a99666d3d7012c2162fdacf84a57fc0b848fd957 Mon Sep 17 00:00:00 2001
 From: Eelco Dolstra <eelco.dolstra@logicblox.com>
 Date: Wed, 16 Apr 2014 10:59:28 +0200
 Subject: [PATCH 03/19] Fix NixOS containers
@@ -10,18 +10,18 @@ container, so checking early whether it exists will fail.
  1 file changed, 2 insertions(+)
 
 diff --git a/src/nspawn/nspawn.c b/src/nspawn/nspawn.c
-index 7515380fcd..14f8a82eb8 100644
+index 04685fecba..0e5ece5f91 100644
 --- a/src/nspawn/nspawn.c
 +++ b/src/nspawn/nspawn.c
-@@ -5323,6 +5323,7 @@ static int run(int argc, char *argv[]) {
+@@ -5590,6 +5590,7 @@ static int run(int argc, char *argv[]) {
                                  goto finish;
                          }
                  } else {
 +#if 0
                          const char *p, *q;
 
                          if (arg_pivot_root_new)
-@@ -5337,6 +5338,7 @@ static int run(int argc, char *argv[]) {
+@@ -5604,6 +5605,7 @@ static int run(int argc, char *argv[]) {
                                  r = -EINVAL;
                                  goto finish;
                          }
@@ -30,5 +30,5 @@ index 7515380fcd..14f8a82eb8 100644
 
          } else {
 -- 
-2.30.1
+2.32.0
 

pkgs/os-specific/linux/systemd/0004-Look-for-fsck-in-the-right-place.patch

@@ -1,4 +1,4 @@
-From 3a721cf70e952e933ef5374006bbb11a3a0ad36a Mon Sep 17 00:00:00 2001
+From 3f0780b25bdbe4156a2f761c90083bbba5f4d473 Mon Sep 17 00:00:00 2001
 From: Eelco Dolstra <eelco.dolstra@logicblox.com>
 Date: Thu, 1 May 2014 14:10:10 +0200
 Subject: [PATCH 04/19] Look for fsck in the right place
@@ -8,7 +8,7 @@ Subject: [PATCH 04/19] Look for fsck in the right place
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/src/fsck/fsck.c b/src/fsck/fsck.c
-index 510689f3b7..25cab5acae 100644
+index cd7adfaeb9..68cebdd158 100644
 --- a/src/fsck/fsck.c
 +++ b/src/fsck/fsck.c
 @@ -368,7 +368,7 @@ static int run(int argc, char *argv[]) {
@@ -21,5 +21,5 @@ index 510689f3b7..25cab5acae 100644
                  cmdline[i++] = "-T";
 
 -- 
-2.30.1
+2.32.0
 

pkgs/os-specific/linux/systemd/0005-Add-some-NixOS-specific-unit-directories.patch

@@ -1,4 +1,4 @@
-From 8b7f881cf22e98e907506f4c403b9e304e332bf9 Mon Sep 17 00:00:00 2001
+From 82698c6a5142e710c302f9c38367ed00d8ec94ba Mon Sep 17 00:00:00 2001
 From: Eelco Dolstra <eelco.dolstra@logicblox.com>
 Date: Fri, 19 Dec 2014 14:46:17 +0100
 Subject: [PATCH 05/19] Add some NixOS-specific unit directories
@@ -10,14 +10,14 @@ units provided by packages installed into the default profile via
 Also, remove /usr and /lib as these don't exist on NixOS.
 ---
  src/basic/path-lookup.c | 17 ++---------------
- src/core/systemd.pc.in  |  5 +++--
- 2 files changed, 5 insertions(+), 17 deletions(-)
+ src/core/systemd.pc.in  |  8 ++++----
+ 2 files changed, 6 insertions(+), 19 deletions(-)
 
 diff --git a/src/basic/path-lookup.c b/src/basic/path-lookup.c
-index 96b82170d0..bf66bd6b77 100644
+index 05eb17d66c..1cd141d012 100644
 --- a/src/basic/path-lookup.c
 +++ b/src/basic/path-lookup.c
-@@ -94,11 +94,7 @@ int xdg_user_data_dir(char **ret, const char *suffix) {
+@@ -91,11 +91,7 @@ int xdg_user_data_dir(char **ret, const char *suffix) {
  }
 
  static const char* const user_data_unit_paths[] = {
@@ -29,7 +29,7 @@ index 96b82170d0..bf66bd6b77 100644
          NULL
  };
 
-@@ -616,15 +612,13 @@ int lookup_paths_init(
+@@ -613,15 +609,13 @@ int lookup_paths_init(
                                          persistent_config,
                                          SYSTEM_CONFIG_UNIT_DIR,
                                          "/etc/systemd/system",
@@ -40,13 +40,13 @@ index 96b82170d0..bf66bd6b77 100644
                                          STRV_IFNOTNULL(runtime_attached),
                                          STRV_IFNOTNULL(generator),
 -                                        "/usr/local/lib/systemd/system",
-                                         SYSTEM_DATA_UNIT_PATH,
+                                         SYSTEM_DATA_UNIT_DIR,
 -                                        "/usr/lib/systemd/system",
 -                                        STRV_IFNOTNULL(flags & LOOKUP_PATHS_SPLIT_USR ? "/lib/systemd/system" : NULL),
                                          STRV_IFNOTNULL(generator_late));
                          break;
 
-@@ -640,14 +634,11 @@ int lookup_paths_init(
+@@ -637,14 +631,11 @@ int lookup_paths_init(
                                          persistent_config,
                                          USER_CONFIG_UNIT_DIR,
                                          "/etc/systemd/user",
@@ -62,23 +62,23 @@ index 96b82170d0..bf66bd6b77 100644
                                          STRV_IFNOTNULL(generator_late));
                          break;
 
-@@ -797,7 +788,6 @@ char **generator_binary_paths(UnitFileScope scope) {
+@@ -794,7 +785,6 @@ char **generator_binary_paths(UnitFileScope scope) {
                  case UNIT_FILE_SYSTEM:
                          add = strv_new("/run/systemd/system-generators",
                                         "/etc/systemd/system-generators",
 -                                       "/usr/local/lib/systemd/system-generators",
                                         SYSTEM_GENERATOR_DIR);
                          break;
 
-@@ -805,7 +795,6 @@ char **generator_binary_paths(UnitFileScope scope) {
+@@ -802,7 +792,6 @@ char **generator_binary_paths(UnitFileScope scope) {
                  case UNIT_FILE_USER:
                          add = strv_new("/run/systemd/user-generators",
                                         "/etc/systemd/user-generators",
 -                                       "/usr/local/lib/systemd/user-generators",
                                         USER_GENERATOR_DIR);
                          break;
 
-@@ -844,12 +833,10 @@ char **env_generator_binary_paths(bool is_system) {
+@@ -841,12 +830,10 @@ char **env_generator_binary_paths(bool is_system) {
                  if (is_system)
                          add = strv_new("/run/systemd/system-environment-generators",
                                          "/etc/systemd/system-environment-generators",
@@ -92,10 +92,10 @@ index 96b82170d0..bf66bd6b77 100644
 
                  if (!add)
 diff --git a/src/core/systemd.pc.in b/src/core/systemd.pc.in
-index b5cc8f94a5..a701cd05f8 100644
+index fc0f8c34fa..ded74ce50a 100644
 --- a/src/core/systemd.pc.in
 +++ b/src/core/systemd.pc.in
-@@ -38,10 +38,11 @@ systemdsystemconfdir=${systemd_system_conf_dir}
+@@ -38,10 +38,10 @@ systemdsystemconfdir=${systemd_system_conf_dir}
  systemd_user_conf_dir=${sysconfdir}/systemd/user
  systemduserconfdir=${systemd_user_conf_dir}
 
@@ -105,10 +105,22 @@ index b5cc8f94a5..a701cd05f8 100644
 
 -systemd_user_unit_path=${systemd_user_conf_dir}:/etc/systemd/user:/run/systemd/user:/usr/local/lib/systemd/user:/usr/local/share/systemd/user:${systemd_user_unit_dir}:/usr/lib/systemd/user:/usr/share/systemd/user
 +systemd_user_unit_path=${systemd_user_conf_dir}:/etc/systemd/user:/nix/var/nix/profiles/default/lib/systemd/user:/run/systemd/user:${systemduserunitdir}
-+
  systemduserunitpath=${systemd_user_unit_path}
 
  systemd_system_generator_dir=${root_prefix}/lib/systemd/system-generators
+@@ -50,10 +50,10 @@ systemdsystemgeneratordir=${systemd_system_generator_dir}
+ systemd_user_generator_dir=${prefix}/lib/systemd/user-generators
+ systemdusergeneratordir=${systemd_user_generator_dir}
+
+-systemd_system_generator_path=/run/systemd/system-generators:/etc/systemd/system-generators:/usr/local/lib/systemd/system-generators:${systemd_system_generator_dir}
++systemd_system_generator_path=/run/systemd/system-generators:/etc/systemd/system-generators:${systemd_system_generator_dir}
+ systemdsystemgeneratorpath=${systemd_system_generator_path}
+
+-systemd_user_generator_path=/run/systemd/user-generators:/etc/systemd/user-generators:/usr/local/lib/systemd/user-generators:${systemd_user_generator_dir}
++systemd_user_generator_path=/run/systemd/user-generators:/etc/systemd/user-generators:${systemd_user_generator_dir}
+ systemdusergeneratorpath=${systemd_user_generator_path}
+
+ systemd_sleep_dir=${root_prefix}/lib/systemd/system-sleep
 -- 
-2.30.1
+2.32.0