upx-4.0.2-armeb_linux/upx hangs when running under qemu-armeb #687
Comments
|
@jreiser Could you please run a quick test and confirm the bug? Thanks! |
|
@markus-oberhumer I will wait until tomorow in hope that the fog of too many proposed patches and actual patches and Pull requests clears from qemu. |
|
CC @hdeller Hi Helge, sorry for bothering you again, but maybe you want to have a look. Or otherwise please connect us to the relevant QEMU team. Thanks! |
so Try another version, this time of Helga's qemu-hppa with upx-fix: So that is what Markus is complaining about. |
|
If you run qemu this way:
QEMU_LOG="in_asm,int" QEMU_LOG_FILENAME=logfile qemu/build/qemu-armeb -strace upx-4.0.2-armeb_linux/upx this-file-does-not-exist
you will get the assembly output into the file "logfile".
Then you can see what qemu executed.
If you add "cpu" to QEMU_LOG, then you even see the CPU registers before each block.
This should give an idea what kind of exception happened...
|
|
After some more testing it seems that qemu-6.1.1 (Alpine Linux 3.15) works. I have no interest further debugging this, closing. |
|
Just in case somebody is interested, here is a current upx armeb build (unpacked, with debug symbols). Works fine with qemu-6, won't even start on qemu >= 7 |
|
On 7/21/23 07:07, Markus F.X.J. Oberhumer wrote:
I have no interest debugging this, closing.
I suggest to keep it open.
I don't have time to look into it very soon, but maybe at some later date (or someone else).
Helge
|
|
Sure, I only closed it because I have no time working on it. |
|
No improvements with qemu-8.1.0-rc1, still hangs. |
|
It's a two-instruction loop, with |
|
John, if you
|
|
btw, what is the expected behaviour?
What is expected? |
|
@hdeller It's supposed to continue after the exception has been caught and printed. Just try any qemu-6 or qemu-5 version. |
|
This seems to be a memory-(instruction)-overwrite, either triggered by qemu or from upx. In qemu-7 I see this instruction: and when running qemu-8 (head) I see: after that of course qemu executes other code paths. |
|
the assembly before that is: it seems blx jumps to [r7], in both qemu versions this is the same address 0xffff0fc0, so this is correct. |
|
Please try the attached upx-static-armeb.zip - this is compiled with debug info. EDIT: this behaves differently because I added an early check for working C++ exceptions, so this now hangs at program startup EDIT 2: qemu-6.2.0 works with this file; qemu 7.1.0, 7.2.4, 8.0 and 8.1 do not |
|
This is how I run it: |
|
Update: I have attached new test files (4 builds), including gcc-11 builds upx-devel-build-armeb-static.zip qemu-6 still works on all of these 4 files but qemu-8.1 now crashes, maybe this can help debugging: $ /usr/local/packages/qemu-user-8.1.0-rc1/bin/qemu-armeb ./upx-devel-build-armeb-static/linux-musl-gcc-11/armeb-linux-musleabi-gcc-11.3.0/debug/upx
qemu: uncaught target signal 4 (Illegal instruction) - core dumped
zsh: illegal hardware instruction (core dumped) /usr/local/packages/qemu-user-8.1.0-rc1/bin/qemu-armebP.S. .tar.xz within .zip file beause of GitHub limitation |
|
Another update: qemu-7.0.0 also does NOT work, so the problem (regression?) was introduced between 6.2.0 and 7.0.0 |
|
Could you run a "git bisect" on qemu between 6.0 and 7.0 ? |
|
Running "git bisect" is not as simple as implied by the documentation "git help bisect". You have to use the compiler and other tools that match the version that you're trying to build. For example, qemu v6.2.0 does not build with |
|
yes, I know. I needed also "--disable-werror" (or something like that). |
|
But I think you need to compare 6.0 (not 6.2) to 7.0, since 6.2 is another branch. |
|
|
|
|
|
working... 5 steps remaining. |
Yes, that *.log info is redundant, IF there were no mistakes in the ~50 step process (12 git versions, 4 steps per version.) |
|
@jreiser Great job! |
|
I'm sure some endianess fixup is missing in this commit for armeb... |
|
CC @rth7680 Richard Henderson, asking for feedback |
|
try this patch: |
|
Yes, patch above works for me as expected with qemu v8-rc: |
|
Please also try the 4 updated test files I posted above: |
|
all of those work too, show some doctest stiuff with SUCCESS, and then finish with failure without hang |
|
Great! ( |
|
Against v8.1.0-rc1: It also seems to me that |
|
No, your patch is wrong. You should not swap the "addr" variable. It's already the virtual address which is the same in the emulator. I think cpsr shouldn't be swapped either, but Richard should know better. |
|
I defer to maintainers; obviously they should know. And just as obviously, they should COMMENT THE CODE about why some references to *env get |
|
@jreiser I'm sure you are aware that endian issues are a nasty problem. The linux kernel started using __be32/__le32 quite some time ago (but I think this needs some extra checker, not sure). In UPX we greatly benefit from using C++ because we can use those nice BE32/LE32 classes. |
|
oldval and newval needs to be swapped, because they are written & read with host-endianess (via the qatomic_cmpxchg__nocheck() function) into the guest memory and the guest expects (for armeb) words stored in big-endian. |
|
Closing, thanks to everyone! |
|
Summary links: |
Commit 7f4f0d9 ("linux-user/arm: Implement __kernel_cmpxchg with host atomics") switched to use qatomic_cmpxchg() to swap a word with the memory content, but missed to endianess-swap the oldval and newval values when emulating an armeb CPU, which expects words to be stored in big endian in the guest memory. The bug can be verified with qemu >= v7.0 on any little-endian host, when starting the armeb binary of the upx program, which just hangs without this patch. Cc: qemu-stable@nongnu.org Signed-off-by: Helge Deller <deller@gmx.de> Reported-by: "Markus F.X.J. Oberhumer" <markus@oberhumer.com> Reported-by: John Reiser <jreiser@BitWagon.com> Closes: upx/upx#687 Message-Id: <ZMQVnqY+F+5sTNFd@p100> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Commit 7f4f0d9 ("linux-user/arm: Implement __kernel_cmpxchg with host atomics") switched to use qatomic_cmpxchg() to swap a word with the memory content, but missed to endianess-swap the oldval and newval values when emulating an armeb CPU, which expects words to be stored in big endian in the guest memory. The bug can be verified with qemu >= v7.0 on any little-endian host, when starting the armeb binary of the upx program, which just hangs without this patch. Cc: qemu-stable@nongnu.org Signed-off-by: Helge Deller <deller@gmx.de> Reported-by: "Markus F.X.J. Oberhumer" <markus@oberhumer.com> Reported-by: John Reiser <jreiser@BitWagon.com> Closes: upx/upx#687 Message-Id: <ZMQVnqY+F+5sTNFd@p100> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> (cherry picked from commit 38dd78c) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
|
@hdeller Please make sure the patch also lands in the qemu |
|
On 8/3/23 12:31, Markus F.X.J. Oberhumer wrote:
@hdeller <https://github.com/hdeller> Please make sure the patch also
lands in the qemu |origin/staging-8.0| branch, it currently is only
commited in origin/staging-7.2 and origin/staging.
I know. We need to see which patch ends up in 8.1, then see if it's possible
to downport. Btw, the debian 8.0 package should have those (temporary) fixes.
Helge
|
|
I'm talking about armeb and this commit ~Markus |
|
On 8/3/23 15:00, Markus F.X.J. Oberhumer wrote:
I'm talking about armeb and this commit
https://gitlab.com/qemu-project/qemu/-/commit/38dd78c41eaf08b490c9e7ec68fc508bbaa5cb1d <https://gitlab.com/qemu-project/qemu/-/commit/38dd78c41eaf08b490c9e7ec68fc508bbaa5cb1d>
which I cannot find in staging-8.0 or Debian.
***@***.*** was CC'ed...
|
Commit 7f4f0d9 ("linux-user/arm: Implement __kernel_cmpxchg with host atomics") switched to use qatomic_cmpxchg() to swap a word with the memory content, but missed to endianess-swap the oldval and newval values when emulating an armeb CPU, which expects words to be stored in big endian in the guest memory. The bug can be verified with qemu >= v7.0 on any little-endian host, when starting the armeb binary of the upx program, which just hangs without this patch. Cc: qemu-stable@nongnu.org Signed-off-by: Helge Deller <deller@gmx.de> Reported-by: "Markus F.X.J. Oberhumer" <markus@oberhumer.com> Reported-by: John Reiser <jreiser@BitWagon.com> Closes: upx/upx#687 Message-Id: <ZMQVnqY+F+5sTNFd@p100> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> (cherry picked from commit 38dd78c) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Commit 7f4f0d9 ("linux-user/arm: Implement __kernel_cmpxchg with host atomics") switched to use qatomic_cmpxchg() to swap a word with the memory content, but missed to endianess-swap the oldval and newval values when emulating an armeb CPU, which expects words to be stored in big endian in the guest memory. The bug can be verified with qemu >= v7.0 on any little-endian host, when starting the armeb binary of the upx program, which just hangs without this patch. Cc: qemu-stable@nongnu.org Signed-off-by: Helge Deller <deller@gmx.de> Reported-by: "Markus F.X.J. Oberhumer" <markus@oberhumer.com> Reported-by: John Reiser <jreiser@BitWagon.com> Closes: upx/upx#687 Message-Id: <ZMQVnqY+F+5sTNFd@p100> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> (cherry picked from commit 38dd78c) Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
UPDATE 1: this is NOT a UPX bug; qemu-armeb seems broken since qemu >= 7.0.0; qemu-6.2.0 and earlier do work
UPDATE 2: the UPX team believes this is a qemu bug (regression) introduced between qemu-6.2 and qemu-7.0, probably by commit https://gitlab.com/qemu-project/qemu/-/commit/7f4f0d9ea87097a31b36bcc52b7368efed35593c
UPDATE 3: the problem has been identified and fixed, and hopefully will be resolved in the next qemu release
UPDATE 4: qemu-8.1.0-rc2 includes the fixes; back-ports to older stable qemu versions are pending
UPDATE 5: also fixed in qemu-8.0.4 and qemu-7.2.5
What's the problem (or question)?
qemu-armeb upx-4.0.2-armeb_linux/upxhangs when throwing a C++ exception, with 100% CPU usageqemu-7.1.0 (Alpine Linux 3.17) HANGS
qemu-7.2.1 (Fedora 38) HANGS
qemu-8 git HEAD with
upx-fixHANGSEarlier QEMU versions (qemu-6 and below) seem to work, so this might be a QEMU regression.
How can we reproduce the issue?
Download and untar https://github.com/upx/upx/releases/download/v4.0.2/upx-4.0.2-armeb_linux.tar.xz
Note that this problem is probably unrelated to UPX as the unpacked executable also hangs.
Hint: see https://github.com/upx/upx/blob/devel/misc/test-qemu-with-podman for some simple support scripts how to test various QEMU versions under Podman
P.S. I stumbled on this while testing the fix for #683
The text was updated successfully, but these errors were encountered: