Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running apps with GDB segfaults #1365

Closed
jonnius opened this issue Feb 11, 2020 · 10 comments
Closed

Running apps with GDB segfaults #1365

jonnius opened this issue Feb 11, 2020 · 10 comments

Comments

@jonnius
Copy link
Contributor

@jonnius jonnius commented Feb 11, 2020

  • Device: any (last tested at least FP2 and Xperia X)
  • Channel: any (last tested at devel and nightly CI builds)
  • Build: any (last tested on CI server build from 6th february)

Steps to reproduce

  • Install any app (commands below are for Webber 0.5.1)
  • Launch app at least once
  • Make system image writable
  • Install gdb and some debug packages:
sudo apt update
sudo apt install gdb libssl1.0.0-dbg
  • Enter a shell (e.g. via adb shell)
  • Run the app with gdb:
gdb -ex run --args /opt/click.ubuntu.com/webber.timsueberkrueb/0.5.1/lib/arm-linux-gnueabihf/bin/webber --desktop_file_hint=/home/phablet/.cache/ubuntu-app-launch/desktop/webber.timsueberkrueb_webber_0.5.1.desktop

Expected behavior

The app starts.

Actual behavior

There is a segfault before the app starts with

Program received signal SIGSEGV, Segmentation fault.
0xf77cadde in ?? () from /lib/ld-linux-armhf.so.3

Backtrace:

#0  0xf77cadde in ?? () from /lib/ld-linux-armhf.so.3
#1  0xf77cadf6 in ?? () from /lib/ld-linux-armhf.so.3
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

after installing libc6-dbg (and libssl1.0.0-dbg to get more informative output), there is just a (recoverable) SIGILL:

Program received signal SIGILL, Illegal instruction.
_armv7_tick () at armv4cpuid.S:94
94	armv4cpuid.S: No such file or directory.

Logfiles and additional information

  • When starting the app the same way without gdb, it works fine:
    /opt/click.ubuntu.com/webber.timsueberkrueb/0.5.1/lib/arm-linux-gnueabihf/bin/webber --desktop_file_hint=/home/phablet/.cache/ubuntu-app-launch/desktop/webber.timsueberkrueb_webber_0.5.1.desktop
  • Other devs confirmed that starting apps with GDB has been working before (may have been back in 2017 or 2018).
  • This bug is the reason we cannot implement on-device-debugging in Clickable
@jonnius jonnius changed the title Running apps with GDB segfaults Running apps with GDB segfaults at libssl Feb 11, 2020
@jonnius

This comment has been minimized.

Copy link
Contributor Author

@jonnius jonnius commented Feb 11, 2020

Just found out, if libc6-dbg is installed, one can continue after the segfault and the app starts. Without that package, it is impossible to continue.

@dobey

This comment has been minimized.

Copy link
Member

@dobey dobey commented Feb 12, 2020

It is not a segfault (SIGSEGV), but illegal instruction (SIGILL), which libssl handles internally and is expected. You need to tell gdb to ignore SIGILL, or just continue with the debug packages installed. There's nothing special to be done here.

@dobey dobey closed this Feb 12, 2020
@jonnius

This comment has been minimized.

Copy link
Contributor Author

@jonnius jonnius commented Feb 12, 2020

Well I missed some info when gathering the logs. But without libc6-dbg installed, it actually segfaults (SIGSEGV). Will upload the log later.

@jonnius

This comment has been minimized.

Copy link
Contributor Author

@jonnius jonnius commented Feb 12, 2020

@dobey I just updated the description. Could you please reopen?

@jonnius jonnius changed the title Running apps with GDB segfaults at libssl Running apps with GDB segfaults Feb 12, 2020
@dobey

This comment has been minimized.

Copy link
Member

@dobey dobey commented Feb 13, 2020

@dobey I just updated the description. Could you please reopen?

What is actually crashing? Is it the app running under gdb, or is it gdb itself? Because this really doesn't make sense, and if it is an issue, it seems like a problem in ld and should be reported against Ubuntu. Can the same issue be reproduced on x86 for example when not installing libc6-dbg there? Does it happen with other commands, for example doing gdb -ex run --args /usr/bin/ls -lh instead?

Also, if the problem is fixed by installing libc6-dbg (libc6 is the package that provides the /lib/ld-linux.so.2, though I'm not sure what provides ld-linux-armhf.so instead), it will be quite difficult to debug, given one won't be able to get a stack trace with proper symbols, since the crash will be gone.

On the subject of gdb working prior to ubports/ubuntu-touch-meta#30 being merged, I think libc6-dbg was in the rootfs too, but we needed room and so removed it. I hope the plan is not for clickable to be installing debs and resizing the rootfs partition and such. That would be bad. Let's get a proper fix and gdbserver included in the rootfs by default,

But I guess either way, I think this is an issue that should be pushed upstream, as I don't think UBports should get sucked into maintaining forks of glibc and/or gcc for example, to fix issues like this.

@jonnius

This comment has been minimized.

Copy link
Contributor Author

@jonnius jonnius commented Feb 13, 2020

GDB also segfaults with other commands like ls. So should I open a bug report at gdb or gcc or glibc then?

I cannot reproduce the issue on my Ubuntu 19.10 machine. Should I set up a VM with 16.04 to test whether gdb crashes for any command? I doubt it.

I could live with having gdbserver and libc6-dbg installed in the system by default. If that is what you would recommend, I will open a MR accordingly.

@dobey

This comment has been minimized.

Copy link
Member

@dobey dobey commented Feb 13, 2020

No, I would not recommend having libc6-dbg installed in the rootfs by default. It is way too large. I think clickable or whatever IDE could instead download, push, and unpack it somewhere under the home directory (perhaps install debug packages inside a libertine container?) and tell gdbserver to find the symbols there instead of normal default expected paths.

As for where the bug should be reported, it seems like https://bugs.launchpad.net/ubuntu/+source/gdb/+bug/1325503 might already cover the issue here. The issue isn't exactly the same, but it is probably very directly related.

@jonnius

This comment has been minimized.

Copy link
Contributor Author

@jonnius jonnius commented Feb 14, 2020

Thanks for the clarification. If I understand that bug report correctly, it is expected behavior for gdb to fail without debug symbols being installed.

I just tried and tested to ship all shared object files from libc6-dbg with the click package of the app (only those with matching arch triplet of course). It does not change anything about the segfault when starting this app via gdb though. Do you know what those *.debug files are for in the package and whether I need them and if yes, where?

@dobey

This comment has been minimized.

Copy link
Member

@dobey dobey commented Feb 14, 2020

it is expected behavior for gdb to fail without debug symbols being installed.

I don't think that's true. It will run the app, and any stack traces will just be mostly useless.

I just tried and tested to ship all shared object files from libc6-dbg with the click package of the app (only those with matching arch triplet of course). It does not change anything about the segfault when starting this app via gdb though. Do you know what those *.debug files are for in the package and whether I need them and if yes, where?

You shouldn't actually distribute these in a click package. I don't know what those specific files are for, no. However, in order for gdb to find the symbols installed into a different path, you'll need to do set solib-search-path to an appropriate value, inside gdb before the run command.

@jonnius

This comment has been minimized.

Copy link
Contributor Author

@jonnius jonnius commented Mar 10, 2020

Thanks for your help. I finally made the Clickable feature that does not need to use apt on the device, nor distribute anything with the click package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.