ck_epoch_section_2 hangs in ck_epoch_synchronize when compiled with LTO

[andreas-schwab]

The writer threads in the ck_epoch_section_2 test hang in the first call
to ck_epoch_synchronize. That happens on both aarch64 and riscv64, and is
independ of the number of threads started.

[michael-grunder]

How are you building with LTO exactly. Just normally like the following?

CFLAGS=-flto LDFLAGS=-flto ./configure
make

Also what compiler and aarch64 hardware exactly? I tried to replicate it with clang on apple silicon (M4) but was not able to.

A callstack showing where it's hanging could also be illuminating.

[andreas-schwab]

Here is the full build log:

https://build.opensuse.org/package/live_build_log/openSUSE:Factory:ARM/ck/standard/aarch64

[andreas-schwab]

It loops in the inner loop of ck_epoch_synchronize_wait, calling
ck_epoch_scan all over again because e_d == delta.

[michael-grunder]

I'll see if I can cause it on a rasberry pi since I'm having no luck with apple silicon. One thing I would mention is that typically you enable LTO both at compile time and link time (CFLAGS and LDFLAGS) although it shouldn't generate incorrect code.

There are also a lot of failed to inline warnings which theoretically is a clue.

[andreas-schwab]

This can easily be reproduced with a cross compiler and qemu:

$ CC=aarch64-linux-gcc CFLAGS="-O2 -flto" PLATFORM=aarch64 ./configure --prefix=/usr
$ make
$ make -C regressions/ck_epoch/validate ck_epoch_section_2
$ regressions/ck_epoch/validate/ck_epoch_section_2 1 1 1
.10 0 9
10 0 9
10 0 9
^C

$ CC=riscv64-linux-gcc CFLAGS="-O2 -flto" PLATFORM=riscv64 ./configure --prefix=/usr
$ make
$ make -C regressions/ck_epoch/validate ck_epoch_section_2
$ regressions/ck_epoch/validate/ck_epoch_section_2 1 1 1
.8 0 7
8 0 7
8 0 7
8 0 7
8 0 7
8 0 7
^C

[michael-grunder]

This still does not replicate for me.

 qemu-aarch64 -L /usr/aarch64-linux-gnu ck_epoch_section_2 1 1 1

...3078577 iterations

Cross-compiled with your provided CFLAGS:

❯ file ck_epoch_section_2
ck_epoch_section_2: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=209e44886463dbcc77102e20b8df21705d1c0532, for GNU/Linux 3.7.0, not stripped

I'm on a debian dual xeon host:

❯ uname -a
Linux hydra 6.1.0-29-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.123-1 (2025-01-02) x86_64 GNU/Linux

More information about the specific system you're on may help. What is the exact version of the compiler? How many physical cores does the box/vm have, etc?

If you attach the built binary I can try to run that locally and also dump the dissassembly.

[andreas-schwab]

Tumbleweed uses 14.3, and it is also reproducible with 15.1.

[michael-grunder]

I was able to reproduce it in a Docker container.

Reproducer repo here

The reproducer doesn't use cross compilation but rather Docker's builtin multiarch support.

Edit: This also trivially reproduces on an aarch64 machine for me (rasberry pi 5).

Edit2: Someone who knows more about the actual epoch logic can comment but it looks like this may be a compiler issue.

I can reproduce this behavior with gcc-14 and gcc-15 when building with LTO enabled, but the bug disappears if I compile with gcc-13. The bug is also not apparent when compiling with clang 20.

[sbahra]

Thanks all! I'll take a look at this Sunday evening. Thanks for the reproducer repository @michael-grunder.

[sbahra]

I'll have to get to this later in the week folks, apologies.