Commit f0b0b67 breaks building GNU Binutils when using ksh93 as /bin/sh #507
Labels
bug
Something is not working
Comments
|
Investigating. |
|
I compared the good and bad binutils build directories with These files have these large missing code blocks. |
|
This is a daunting problem. I can see three possibilities. The optimisation introduced by f0b0b67 could:
I think 1 is unlikely. I think the optimisation is theoretically and conceptually sound. Of course I could be missing something. But the problem could also easily be number 2 or 3. In case 2, reverting it would just paper over the real ksh bug, and that's not what we're about here. In case 3, we are not the ones who should be changing. So just blindly reverting the optimisation does not seem like a good idea. I really, really need a minimal reproducer to decide what to do about this issue. Without one we have no hope of knowing what the bug actually is. But it's very hard to derive one, as the GNU build scripts are highly complicated. |
|
The bug is triggered in binutil's ld/genscripts.sh and scripts sourced by it. This patch to binutils allows tracing the execution path of these scripts in the `make` output (click the arrow to expand).diff -ur binutils-2.35.2.orig/ld/genscripts.sh binutils-2.35.2/ld/genscripts.sh
--- binutils-2.35.2.orig/ld/genscripts.sh 2021-01-30 09:38:04.000000000 +0100
+++ binutils-2.35.2/ld/genscripts.sh 2022-08-05 07:02:54.940660426 +0200
@@ -1,4 +1,10 @@
#!/bin/sh
+case $KSH_VERSION in
+Version*)
+ PS4='+ [${.sh.subshell:+S${.sh.subshell},}${.sh.file:+${.sh.file#${.sh.file%/*/*}/},}${.sh.fun:+${.sh.fun},}${LINENO:+L$LINENO,}e$?] '
+ set -x
+ ;;
+esac
# genscripts.sh - generate the ld-emulation-target specific files
# Copyright (C) 2004-2020 Free Software Foundation, Inc.
#As a way of finding where it goes wrong, we can then diff the make output of a bad build with that of a good build. Here are those results from my testing. That diff does not indicate a bug in the GNU scripts. Clearly, code is not getting executed that should be. Specifically, it looks like the |
|
I have not found a minimal reproducer yet. And it looks like the problem is actually with the optimisation I introduced. It's not another bug in ksh, it was all me. :-/ More research is needed, so I'll just have to revert it. I'll make an urgent 1.0.1 point release as this is a fatal bug. |
McDutchie
added a commit
that referenced
this issue
Aug 5, 2022
As of 2022-06-18, ksh 93u+m is not capable of being used as /bin/sh while building GNU binutils. The execution of some of its build system's dot scripts is incorrectly aborted as an external 'sed' command is execve(2)'d without forking. This means that incorrect exec optimization was happening. Unfortunately I have not been able to derive a minimal reproducer of the problem yet because the GNU binutils build scripts are very complex. Pending further research, the optimisation is reverted. Even if a way to make it work is found, it will not be reintroduced to the 1.0 branch. Thanks to @atheik for finding the problem and identifying the commit that introduced it. Resolves: #507
McDutchie
added a commit
that referenced
this issue
Aug 5, 2022
As of 2022-06-18, ksh 93u+m is not capable of being used as /bin/sh while building GNU binutils. The execution of some of its build system's dot scripts is incorrectly aborted as an external 'sed' command is execve(2)'d without forking. This means that incorrect exec optimization was happening. Unfortunately I have not been able to derive a minimal reproducer of the problem yet because the GNU binutils build scripts are very complex. Pending further research, the optimisation is reverted. Even if a way to make it work is found, it will not be reintroduced to the 1.0 branch. Thanks to @atheik for finding the problem and identifying the commit that introduced it. Resolves: #507
McDutchie
added a commit
that referenced
this issue
Aug 5, 2022
As of 2022-06-18, ksh 93u+m is not capable of being used as /bin/sh while building GNU binutils. The execution of some of its build system's dot scripts is incorrectly aborted as an external 'sed' command is execve(2)'d without forking. This means that incorrect exec optimization was happening. Unfortunately I have not been able to derive a minimal reproducer of the problem yet because the GNU binutils build scripts are very complex. Pending further research, the optimisation is reverted. Even if a way to make it work is found, it will not be reintroduced to the 1.0 branch. Thanks to @atheik for finding the problem and identifying the commit that introduced it. Resolves: #507
|
@McDutchie, thank you for the thorough analysis! cat <<'EOF1' >A
cat <<'EOF2' >B
( echo B1 ) | cat
( echo B2 ) | cat
EOF2
cat <<'EOF2' >C
( echo C1 ) | cat
( echo C2 ) | cat
EOF2
(
. ./B
. ./C
) | cat
EOF1
git checkout f0b0b67
rm -rf arch/$(bin/package host)
bin/package make
mv arch/$(bin/package host) build-f0b0b67
build-f0b0b67/bin/ksh ./A
# B1
# B2
git checkout 2124f14
rm -rf arch/$(bin/package host)
bin/package make
mv arch/$(bin/package host) build-2124f14
build-2124f14/bin/ksh ./A
# B1
# B2
# C1
# C2
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The version of GNU Binutils being built doesn't seem to make a difference.
The build succeeds but is completely broken.
If you would like, you can reproduce this in Debian Live under x86_64 QEMU:
Run the following in Debian Live:
Sorry, I know this makes for a pretty poor reproducer.
I'll try to look into this further.
I ran into this issue during a build of a Linux system that uses ksh93 as
/bin/sh. #467 and #470 were found similarly a while back. I recently hadn't updated the system so I unfortunately missed the week-long testing of93u+m/1.0.0-rc.1.The text was updated successfully, but these errors were encountered: