Add DWARF unwinding, and an external debug info loader for ELF #15823
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kcbanner
force-pushed
the
dwarf_unwind
branch
7 times, most recently
from
81f9ece
to
c99a436
Compare
kcbanner
force-pushed
the
dwarf_unwind
branch
2 times, most recently
from
4cc7a33
to
6d4b19b
Compare
kcbanner
force-pushed
the
dwarf_unwind
branch
9 times, most recently
from
bdf575a
to
8505ff0
Compare
kcbanner
force-pushed
the
dwarf_unwind
branch
6 times, most recently
from
7d3a510
to
d9a6fdf
Compare
14 tasks
Renamed dwarf_unwinding -> stack_iterator to better reflect that it's not just DWARF unwinding. Added a test for unwinding with a frame pointer.
debug: handle the possibility of eh_frame / debug_frame being mapped in memory or loaded from disk
…m the __unwind_info directly
… itself is updated by a column rule
dwarf: handle signal frame CIE flag
dwarf: fixup unchecked .eh_frame CIE offset subtraction
andrewrk
reviewed
Jul 24, 2023
Comment on lines
+416
to
+422
| pub usingnamespace if (builtin.os.tag == .linux and builtin.target.isMusl()) struct { | ||
| // musl does not implement getcontext | ||
| pub const getcontext = std.os.linux.getcontext; | ||
| } else struct { | ||
| pub extern "c" fn getcontext(ucp: *std.os.ucontext_t) c_int; | ||
| }; | ||
|
|
There was a problem hiding this comment.
No need for this usingnamespace. Just put the definition here and let the linker tell the user that the symbol does not exist. The user should do this logic themselves if they want to reach into Linux directly.
This is the std.c namespace after all, it should not pretend to be libc and then do something else.
|
I'm seeing massive compilation speed regression in compiling hello world from these changes. Can you double check your usage of comptime and try to reduce the harm here? If it's needed I can work on |
|
I did a small analysis. It seems that the binary size of an empty program grew by over 400 kB. Expand |
|
@IntegratedQuantum thanks! Indeed, reducing those cases does improve the compile time, working on those fixes here: https://github.com/kcbanner/zig/tree/fixup_unwind_perf_regression |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This supercedes (and includes the changes from) #15531.
These changes were developed in tandem with this PR, which might be a good reference for reviewers: kubkon/zig-dwarfdump#1
Closes #13114
Stack unwinding using DWARF unwind info
This change allows stacks to be unwound (and thus stack traces printed) for code that is compiled without
-fomit-frame-pointer. This is accomplished by utilizing the DWARF debug info that is written into the.eh_framesection (or.debug_framesection). Conceptually, those sections contain a large lookup table mapping program addresses to a set of rules per-register that allow you to recover the value of that register in the previous frame - this allows you to recover the return address and previous stack frame values. Instead of actually storing the full lookup table (which would be large), it's stored as a set of instructions (not native code, but a special DWARF CFI bytecode), that allow you to build this table as needed. This system is howlibunwindand debuggers themselves can unwind stacks.This system is described in section 6.4.1 of this document: https://dwarfstd.org/doc/DWARF5.pdf
The overall motivation for this PR is to have zig's stack trace output be at least as good as
gdb.Changes:
.eh_frame,.eh_frame_hdr, and.debug_frame. I've utilizedzig-dwarfdumpas a testbed for the parsers (see linked PR above), and it's able to output an exact match ofllvm-dwarfdumpfor Ubuntu's libc currently.dwarf.call_frame.VirtualMachinewhich can run the CFI instructions and build the aforementioned register-rule lookup table.debug.StackIteratorto utilize the DWARF debug info to unwind the stack, if it is available. It falls back to standard FP-based unwinding if it encounters any errors or missing info. I've tried to have the unwinder fail gracefully as much as possible. For example, a bad subtraction or overflow while running the DWARF opcodes (potentially due to malformed debug info) results in an error and fallback to FP unwinding instead of panicing while panicing.DW_AT_rangesin subprograms. Some DWARF5 subprograms have non-contiguous instruction ranges. An example of such a function isputsin Ubuntu's libc. Stack traces now include the names of these functions.getcontextimplementations forx86-linuxandx86_64-linux. If linkinglibc, then the system provided implementation is used, with the exception of-muslwhich doesn't providegetcontext. These are used to get a register context to unwind stacks via a@panic(vs a segfault handler, where the OS provides the context). Support for additional platforms is straightforward to add and once this is merged I want to open contributor-friendly issues to add support for the other major platforms that I don't have access to.An example of how this improves the output, in this case calling
putswith a bad pointer:Before:
After:
Output when unwinding stack beginning in a
libcwithout frame pointers, on a system that doesn't have debug info installed for libc (in this example, running an x86 program when there is only x86_64 libc symbols available). Note that the unwound stack skips several frames (due to missing unwind info), but the user is notified of this:Stack unwinding on Darwin using
__unwind_infoApple has their own proprietary unwind info format: https://faultlore.com/blah/compact-unwinding/. It works in a similar way to the DWARF unwind tables, but it's a bit less general.
StackIteratorwill attempt to use this unwind info on macos, if it's available. The format supports falling back to DWARF unwinding, which is also supported in this implementation.If only DWARF information is available then normal FP stack unwinding is used, because in this case I observed that the DWARF unwind info for some functions was incorrect. Specifically, certain FDEs would have instruction ranges that spanned functions that they shouldn't have.
On
aarch64, Apple requires that the frame pointer be used. Onx86_64, this isn't the case.Now that we support unwind tables, I've updated
target.needUnwindTablesto return true forofmt = .macho.External debug info
Some distributions (ie. Ubuntu) have their libc debug info in separate files. This scheme allows for shipping debug info separately, which, can save space if the user doesn't actually need to debug the program / library. I've added support for reading external debug info:
.gnu_debuglinksection to look up external debug infoAs an example, in the stack traces above, their debug info actually comes from
/usr/lib/debug/.build-id/69/389d485a9793dbe873f0ea2c93e02efaa9aa3d.debug.Other changes
-gdwarf). I added a test to cover this scenario.Remaining Work
This is a work in progress, but it's at the point where I'd like to get it running against the CIs for arches / OSes I don't have access to, so this will be in draft status until these items are complete:
__unwind_infosupport)Support aarch64-macos (add. Deferring this to a future PR. Since Apple mandates that the FP be used on this platform, most traces will unwind fine with the FP-based StackIterator. I ran into what I suspect is a bug in the CFI generation on this platform, but haven't confirmed this yet.__unwind_infosupport).eh_frame_hdrsection. This is an optimization that will speed up unwinding - it contains a sorted lookup table for the information in.eh_frame, which is binary searchable.eh_frameandeh_frame_hdrare already mapped into the program's address space). Support using these already-mapped sections directly. This is particularly important for things that just measure stack traces and don't need to write them yet (ie. GPA capturing backtraces - with this improvement it could support-fomit-frame-pointertraces without touching the disk)..debug_frame. This is a very similar section to.eh_framewith some slight changes.Update all uses in std of. I'm going to address this in a future PR. I've addressed all uses except forStackIteratorto use the DWARF unwinding version, if availablecaptureStackTrace, which doesn't have access to an allocator, which is currently required for the unwinder.Thanks to @jacobly0 for the help debugging aarch64-macos, and @kubkon for answering all my linker questions!