clang miscompiles systemd on ppc64le

[Gelbpunkt]

Hi there, I'm building systemd with clang on several architectures and setups, and it works great on almost all of them, but not all.

On Gentoo Linux (ppc64) with clang 21.1.7 and on Alpine Linux (ppc64le) with clang 21.1.7, it seems like clang miscompiles libsystemd-shared-25X.so (I'm building a patched systemd 257 on Alpine and unpatched vanilla 259_rc3 on Gentoo).

The system will subsequently crash during boot with segfaults in libsystemd-shared-25X.so during boot. Here's such a log:

Log of Alpine ppc64le with systemd 257 build with clang 21.1.7

[    2.117601] systemd[1]: systemd 257.10 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE -TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF +XKBCOMMON -UTMP -SYSVINIT -LIBARCHIVE)
[    2.118991] systemd[1]: Detected virtualization kvm.
[    2.119294] systemd[1]: Detected architecture ppc64-le.

Welcome to postmarketOS edge!

[    2.121730] systemd[1]: Hostname set to <amadioha>.
[    2.129983] systemd[1]: Failed to open libbpf, cgroup BPF features disabled: Not supported
[    2.286354] systemd[1]: Queued start job for default target Graphical Interface.
[    2.306785] systemd[1]: Created slice Slice /system/getty.
[  OK  ] Created slice Slice /system/getty.
[    2.309134] systemd[1]: Created slice Slice /system/modprobe.
[  OK  ] Created slice Slice /system/modprobe.
[    2.312187] systemd[1]: Created slice Slice /system/serial-getty.
[  OK  ] Created slice Slice /system/serial-getty.
[    2.315515] systemd[1]: Created slice User and Session Slice.
[  OK  ] Created slice User and Session Slice.
[    2.318080] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[    2.320901] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[    2.324473] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[  OK  ] Set up automount Arbitrary Executa…ormats File System Automount Point.
[    2.327929] systemd[1]: Expecting device /dev/hvc0...
         Expecting device /dev/hvc0...
[    2.329310] systemd[1]: Reached target Local Encrypted Volumes.
[  OK  ] Reached target Local Encrypted Volumes.
[    2.331729] systemd[1]: Reached target Local Integrity Protected Volumes.
[  OK  ] Reached target Local Integrity Protected Volumes.
[    2.334405] systemd[1]: Reached target Path Units.
[  OK  ] Reached target Path Units.
[    2.336130] systemd[1]: Reached target Remote Encrypted Volumes.
[  OK  ] Reached target Remote Encrypted Volumes.
[    2.338583] systemd[1]: Reached target Remote File Systems.
[  OK  ] Reached target Remote File Systems.
[    2.340971] systemd[1]: Reached target Slice Units.
[  OK  ] Reached target Slice Units.
[    2.342771] systemd[1]: Reached target Local Verity Protected Volumes.
[  OK  ] Reached target Local Verity Protected Volumes.
[    2.347991] systemd[1]: Listening on Process Core Dump Socket.
[  OK  ] Listening on Process Core Dump Socket.
[    2.352201] systemd[1]: Listening on Credential Encryption/Decryption.
[  OK  ] Listening on Credential Encryption/Decryption.
[    2.354932] systemd[1]: Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket (/dev/log).
[    2.357499] systemd[1]: Listening on Journal Sockets.
[  OK  ] Listening on Journal Sockets.
[    2.359619] systemd[1]: Listening on DDI File System Mounter Socket.
[  OK  ] Listening on DDI File System Mounter Socket.
[    2.362013] systemd[1]: Listening on Network Service Netlink Socket.
[  OK  ] Listening on Network Service Netlink Socket.
[    2.364629] systemd[1]: Listening on Namespace Resource Manager Socket.
[  OK  ] Listening on Namespace Resource Manager Socket.
[    2.367063] systemd[1]: Userspace Out-Of-Memory (OOM) Killer Socket was skipped because of an unmet condition check (ConditionPathExists=/proc/pressure/memory).
[    2.367786] systemd[1]: Listening on udev Control Socket.
[  OK  ] Listening on udev Control Socket.
[    2.369381] systemd[1]: Listening on udev Kernel Socket.
[  OK  ] Listening on udev Kernel Socket.
[    2.371471] systemd[1]: Huge Pages File System was skipped because of an unmet condition check (ConditionPathExists=/sys/kernel/mm/hugepages).
[    2.394963] systemd[1]: Mounting POSIX Message Queue File System...
         Mounting POSIX Message Queue File System...
[    2.397972] systemd[1]: Mounting Kernel Debug File System...
         Mounting Kernel Debug File System...
[    2.400234] systemd[1]: Mounting Kernel Trace File System...
         Mounting Kernel Trace File System...
[    2.403912] systemd[1]: Starting Create List of Static Device Nodes...
         Starting Create List of Static Device Nodes...
[    2.408515] systemd[1]: Starting Load Kernel Module configfs...
         Starting Load Kernel Module configfs...
[    2.412647] systemd[1]: Starting Load Kernel Module dm_mod...
         Starting Load Kernel Module dm_mod...
[    2.417470] systemd[1]: Starting Load Kernel Module drm...
         Starting Load Kernel Module drm...
[    2.421982] systemd[1]: Starting Load Kernel Module efi_pstore...
         Starting Load Kernel Module efi_pstore...
[    2.429034] systemd[1]: Starting Load Kernel Module fuse...
         Starting Load Kernel Module fuse...
[    2.433279] systemd[1]: Starting Load Kernel Module loop...
         Starting Load Kernel Module loop...
[    2.435941] device-mapper: uevent: version 1.0.3
[    2.436530] systemd[1]: Clear Stale Hibernate Storage Info was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67).
[    2.436644] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev
[    2.441432] systemd[1]: Starting Journal Service...
         Starting Journal Service...
[    2.446231] systemd[1]: Starting Load Kernel Modules...
         Starting Load Kernel Modules...
[    2.449903] systemd[1]: Starting Generate network units from Kernel command line...
         Starting Generate network units from Kernel command line...
[    2.453414] systemd[1]: Starting Remount Root and Kernel File Systems...
         Starting Remount Root and Kernel File Systems...
[    2.458574] fuse: init (API version 7.45)
[    2.459310] systemd[1]: Starting Load udev Rules from Credentials...
         Starting Load udev Rules from Credentials...
[    2.463107] systemd[1]: Starting Coldplug All udev Devices...
         Starting Coldplug All udev Devices...
[    2.466226] loop: module loaded
[    2.466335] systemd[1]: Starting swap with zram...
         Starting swap with zram...
[    2.470578] systemd[1]: Mounted POSIX Message Queue File System.
[  OK  ] Mounted POSIX Message Queue File System.
[    2.472629] systemd[1]: Mounted Kernel Debug File System.
[  OK  ] Mounted Kernel Debug File System.
[    2.475547] systemd[1]: Mounted Kernel Trace File System.
[  OK  ] Mounted Kernel Trace File System.
[    2.477864] systemd[1]: Finished Create List of Static Device Nodes.
[  OK  ] Finished Create List of Static Device Nodes.
[    2.479794] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[    2.480336] systemd[1]: Finished Load Kernel Module configfs.
[  OK  ] Finished Load Kernel Module configfs.
[    2.482454] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[    2.483037] systemd[1]: Finished Load Kernel Module dm_mod.
[  OK  ] Finished Load Kernel Module dm_mod.
[    2.485112] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[    2.485835] systemd[1]: Finished Load Kernel Module efi_pstore.
[  OK  ] Finished Load Kernel Module efi_pstore.
[    2.487796] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[    2.488316] systemd[1]: Finished Load Kernel Module fuse.
[  OK  ] Finished Load Kernel Module fuse.
[    2.490550] systemd[1]: modprobe@drm.service: Deactivated successfully.
[    2.491073] systemd[1]: Finished Load Kernel Module drm.
[  OK  ] Finished Load Kernel Module drm.
[    2.493682] systemd[1]: modprobe@loop.service: Deactivated successfully.
[    2.494180] systemd[1]: Finished Load Kernel Module loop.
[  OK  ] Finished Load Kernel Module loop.
[    2.495948] systemd[1]: Finished Load Kernel Modules.
[  OK  ] Finished Load Kernel Modules.
[    2.501483] systemd[1]: Mounting FUSE Control File System...
         Mounting FUSE Control File System...
[    2.504322] EXT4-fs (vda2): re-mounted 84f283c0-5eb0-4451-857f-832485a7d57b.
[    2.506688] systemd[1]: Mounting Kernel Configuration File System...
         Mounting Kernel Configuration File System...
[    2.508175] systemd[1]: Repartition Root Disk was skipped because no trigger condition checks were met.
[    2.511671] systemd[1]: Starting Apply Kernel Variables...
         Starting Apply Kernel Variables...
[    2.515344] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...
         Starting Create Static Device Nodes in /dev gracefully...
[    2.519875] systemd[1]: Finished Generate network units from Kernel command line.
[  OK  ] Finished Generate network units from Kernel command line.
[    2.524689] systemd[1]: Finished Remount Root and Kernel File Systems.
[  OK  ] Finished Remount Root and Kernel File Systems.
[    2.527902] systemd[1]: Finished Load udev Rules from Credentials.
[  OK  ] Finished Load udev Rul[    2.528916] systemd-journald[960]: Collecting audit messages is disabled.
es from Credentials.
[    2.530942] systemd[1]: Mounted FUSE Control File System.
[  OK  ] Mounted FUSE Control File System.
[    2.534140] systemd[1]: Mounted Kernel Configuration File System.
[  OK  ] Mounted Kernel Configuration File System.
[    2.536065] systemd[1]: Reached target Preparation for Network.
[  OK  ] Reached target Preparation for Network.
[    2.538264] systemd[1]: Rebuild Hardware Database was skipped because of an unmet condition check (ConditionNeedsUpdate=/etc).
[    2.538802] systemd[1]: Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[    2.540893] systemd[1]: Starting Load/Save OS Random Seed...
         Starting Load/Save OS Random Seed...
[    2.546959] zram: Added device: zram0
[    2.549243] zram: Added device: zram1
[    2.549566] zram: Added device: zram2
[    2.550923] systemd[1]: Finished Apply Kernel Variables.
[  OK  ] Finished Apply Kernel Variables.
[    2.553099] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.
[  OK  ] Finished Create Static Device Nodes in /dev gracefully.
[    2.555352] systemd[1]: Create System Users was skipped because no trigger condition checks were met.
[    2.555915] systemd[1]: Userspace Out-Of-Memory (OOM) Killer was skipped because of an unmet condition check (ConditionPathExists=/proc/pressure/memory).
[    2.559439] systemd[1]: Starting Network Time Synchronization...
         Starting Network Time Synchronization...
[    2.562640] systemd[1]: Starting Create Static Device Nodes in /dev...
         Starting Create Static Device Nodes in /dev...
[    2.564826] systemd[1]: Started Journal Service.
[  OK  ] Started Journal Service.
[    2.581613] zram0: detected capacity change from 0 to 16715776
[  OK  ] Finished Create Static Device Nodes in /dev.
[  OK  ] Finished Coldplug All udev Devices.
[  OK  ] Finished Load/Save OS Random Seed.
[    2.612420] Adding 8357824k swap on /dev/zram0.  Priority:16383 extents:1 across:8357824k SSDsc
[  OK  ] Finished swap with zram.
[  OK  ] Reached target Preparation for Local File Systems.
[  OK  ] Reached target Swaps.
         Starting Flush J[    2.644192] sournal to Persisystemd-journald[t960]: Received client request toent Storage. flush runtime j.ournal.
.
         Starting Rule-based Manager for Device Events and Files...
[    2.650546] systemd-journald[960]: File /var/log/journal/d036eaa3cf004b13ac458e22db8c94e4/system.journal corrupted or uncleanly shut down, renaming and replacing.
[  OK  ] Started Network Time Synchronization.
[    2.676422] systemd[1]: segfault (11) at 6e41600065636976 nip 7fffb64f6d68 lr 7fffb64f6c5c code 3 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.677378] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.678129] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.737595] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.738650] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.739133] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.743068] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.743511] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.743829] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.745705] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.746179] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.746495] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.748934] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.749355] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.749662] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.754886] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.755302] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.755609] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.758761] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.759247] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.759543] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.763989] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.764527] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.764955] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.767774] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.768455] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.768932] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.771215] systemd[1]: segfault (11) at 5 nip 7fffb64f6d68 lr 7fffb64f6c5c code 1 in libsystemd-shared-257.so[266d68,7fffb6370000+2d0000]
[    2.771911] systemd[1]: code: 5786077e 5787ee7e 6a5b0001 7f84899e 7f44b19e 6a110001 7e447d9e 7e04b59e
[    2.772328] systemd[1]: code: 28340000 7c8f2378 7c8e2378 41820020 <89f40000> e88101b0 38a58e68 71ef00ff
[    2.806281] rtc-generic rtc-generic: registered as rtc0
[    2.806885] rtc-generic rtc-generic: setting system clock to 2025-12-15T05:43:21 UTC (1765777401)
[    2.815001] virtio_scsi virtio5: 32/0/0 default/read/poll queues
[    2.831115] usbcore: registered new interface driver usbfs
[    2.831377] usbcore: registered new interface driver hub
[    2.831611] usbcore: registered new device driver usb
[    2.835253] virtio_net virtio0 enp0s1: renamed from eth0
[    2.850233] xhci_hcd 0000:00:02.0: xHCI Host Controller
[    2.850468] xhci_hcd 0000:00:02.0: new USB bus registered, assigned bus number 1
[    2.851309] xhci_hcd 0000:00:02.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    2.851576] xhci_hcd 0000:00:02.0: lsa_required: 0, lsa_enabled: 0, direct mapping: 1
[    2.852828] xhci_hcd 0000:00:02.0: hcc params 0x00087001 hci version 0x100 quirks 0x0000000000000010
[    2.858430] xhci_hcd 0000:00:02.0: xHCI Host Controller
[    2.858587] xhci_hcd 0000:00:02.0: new USB bus registered, assigned bus number 2
[    2.858831] xhci_hcd 0000:00:02.0: Host supports USB 3.0 SuperSpeed
[    2.859515] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18
[    2.864310] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    2.864685] usb usb1: Product: xHCI Host Controller
[    2.864884] usb usb1: Manufacturer: Linux 6.18.0-6-virt xhci-hcd
[    2.865096] usb usb1: SerialNumber: 0000:00:02.0
[    2.865638] hub 1-0:1.0: USB hub found
[    2.866011] hub 1-0:1.0: 4 ports detected
[    2.867224] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    2.867649] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.18
[    2.867878] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    2.868103] usb usb2: Product: xHCI Host Controller
[    2.868254] usb usb2: Manufacturer: Linux 6.18.0-6-virt xhci-hcd
[    2.868430] usb usb2: SerialNumber: 0000:00:02.0
[    2.869583] hub 2-0:1.0: USB hub found
[    2.869883] hub 2-0:1.0: 4 ports detected
[    2.898284] scsi host0: Virtio SCSI HBA
[    3.554124] systemd[1]: bad frame in handle_rt_signal64: 00000000737be27b nip 00007fffb64f66ac lr 000000013e9324a0
[    3.555116] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000008b
[    3.555392] CPU: 24 UID: 0 PID: 1 Comm: systemd Not tainted 6.18.0-6-virt #7-Alpine NONE
[    3.555651] Hardware name: IBM pSeries (emulated by qemu) POWER8 (architected) 0x4b0201 0xf000004 of:SLOF,HEAD hv:linux,kvm pSeries
[    3.556101] Call Trace:
[    3.556197] [c0000000038bfa20] [c000000000ee2764] dump_stack_lvl+0x88/0xdc (unreliable)
[    3.556481] [c0000000038bfa50] [c000000000113afc] vpanic+0x130/0x3ec
[    3.556722] [c0000000038bfaf0] [c000000000113df0] arch_set_bit+0x0/0x50
[    3.556954] [c0000000038bfb10] [c00000000011dc24] do_exit+0xba4/0xbb0
[    3.557199] [c0000000038bfbe0] [c00000000011de88] do_group_exit+0x38/0xd0
[    3.557434] [c0000000038bfc30] [c000000000137d64] get_signal+0xc34/0xc70
[    3.557670] [c0000000038bfd10] [c000000000021d0c] do_notify_resume+0x11c/0x4e0
[    3.557934] [c0000000038bfdd0] [c000000000030530] interrupt_exit_user_prepare_main+0x170/0x260
[    3.558244] [c0000000038bfe20] [c0000000000308d4] interrupt_exit_user_prepare+0x54/0xc0
[    3.558512] [c0000000038bfe50] [c00000000000c974] interrupt_return_srr_user+0x8/0x138
[    3.558794] ---- interrupt: 301 at 0x7fffb64f66ac
[    3.558961] NIP:  00007fffb64f66ac LR: 000000013e9324a0 CTR: 00007fffb64f668c
[    3.559278] REGS: c0000000038bfe80 TRAP: 0301   Not tainted  (6.18.0-6-virt)
[    3.559547] MSR:  800000000000d033 <SF,EE,PR,ME,IR,DR,RI,LE>  CR: 44822242  XER: 00000000
[    3.559836] CFAR: 000000013e936e0c DAR: 00007fffdfdfffe0 DSISR: 42000000 IRQMASK: 0
[    3.559836] GPR00: 00007fffdfe00140 00007fffdfe00140 00007fffb66ae510 0000000000000000
[    3.559836] GPR04: 0000000000000000 000000013e918637 0000000000000074 000000013e916b3e
[    3.559836] GPR08: 000000013e916bad 00007fffb6357ada 0000000000000005 ffffffffffffffe4
[    3.559836] GPR12: 0000000000880000 00007fffb6a75fb8 00007fffb6357378 00007fffb6357378
[    3.559836] GPR16: 00007fffb6357378 0000000000000000 00007fffb6357378 00007fffb6357378
[    3.559836] GPR20: 0000000000000005 00007fffb6357378 00007fffb6336c1d 0000000000000000
[    3.559836] GPR24: 00007fffb6336c1c 00007fffb6357378 00007fffb6336c1d 0000000010000001
[    3.559836] GPR28: 0000000000000a90 0000000000000005 000000000000000b 00007fffdfe01650
[    3.562142] NIP [00007fffb64f66ac] 0x7fffb64f66ac
[    3.562303] LR [000000013e9324a0] 0x13e9324a0
[    3.562490] ---- interrupt: 301
[    3.635282] pstore: backend (nvram) writing error (-1)

The Gentoo logs are near identical. I'm using LLD on Gentoo and mold on Alpine, so it's not linker-related.

The reason why I think this is a miscompilation is that building with -Dmode=developer set does not result in crashes, while -Dmode=release triggers these problems (and it works fine on other architectures).

The library from the crash above can be found here, the same one but built with -Dmode=developer (and therefore not crashing) can be found here.

Usually I'd throw a program in gdb to see what's wrong and grab a backtrace, but since this is systemd I'm not really sure how to debug it. If anyone has tips, that'd be greatly appreciated.

I'd also be happy to provide LLVM developers with SSH/libvirt access to the systems in question for debugging if it helps.

I did notice that the first line of the logs is segfault (11) at 6e41600065636976, which is interesting because that's a string:

6e 41 60 00 65 63 69 76
 n  A  `  \0 e  c  i  v

[thesamesam]

Usually I'd throw a program in gdb to see what's wrong and grab a backtrace, but since this is systemd I'm not really sure how to debug it. If anyone has tips, that'd be greatly appreciated.

If you haven't already, I suggest trying two things:

1. Seeing if any of the testsuite fails. It'll be far easier to debug that.
2. See if systemd-nspawn -b /path/to/some/root results in a crashing just-spawned systemd process. You can then debug it from outside.

You can also use the traditional bisection method of objects, even if it'll be laborious if you can only reproduce by rebooting.

[Gelbpunkt]

Seeing if any of the testsuite fails. It'll be far easier to debug that.

The testsuite passes fine for me on Alpine - I was misremembering, we don't run it. Will give it a shot. I'll check nspawn later, thanks for the tip!

[Gelbpunkt]

The systemd testsuite on 257 does not seem to like musl that much yet, on Gentoo with 259-rc3 it mostly passes:

Summary of Failures:

1212/1747 systemd:test / test-log                                                  FAIL              1.02s   killed by signal 11 SIGSEGV

Ok:                1724
Fail:              1
Skipped:           22

and that one test fail does not seem related to the issue here (I think this may just be another musl-related fail).

[Gelbpunkt]

I was wrong! In a developer build, that test also passes fine. I'll try to run that one test in GDB later.