Make backtrace code read the dSYM data

[dlangBugzillaToGithub]

Mathias LANG (@Geod24) reported this on 2020-01-16T08:30:19Z

Transferred from https://issues.dlang.org/show_bug.cgi?id=20510

CC List

• Walter Bright (@WalterBright)
• Johan Engelen (@JohanEngelen)
• Sönke Ludwig (@s-ludwig)

Description

Currently, on Mac OSX, the linker will strip debug informations to speed up linkage and reduce binary size. Links to the object files are created, and can be read by the debugger. It's also possible to create a .dSYM bundle to distribute alongside the binary.

This leads to the unfortunate consequence that the backtrace code in Druntime, which relies on the `.debug_lines` DWARF section, will not print file/line information in a stack trace, even when compiled with `-g`.

There are workarounds for this:
- DMD preserves the `__debug_line` by marking it as a regular section: https://github.com/dlang/dmd/commit/2bf7d0db29416eacbb01a91e6502140e354ee0ef
- LDC does the same, but had to patch LLVM for that: https://github.com/ldc-developers/llvm-project/commit/110deda1bc1cf195983fea8c1107886057987955

However, this is a hack and has drawbacks. DMD emits fairly poor debug informations but will show file/line on Exceptions stack traces on OSX, while LDC emits proper information but has to be linked against a patched LLVM.

The requirement for a patched LLVM might throw off packagers, e.g. the Homebrew package of LDC does *not* link against that custom LLVM, hence the workaround does not work: https://github.com/ldc-developers/ldc/issues/3280

What would be best, long term, would be for Druntime to read the same `.dSym` data.

[dlangBugzillaToGithub]

pro.mathias.lang (@Geod24) commented on 2020-04-19T08:30:00Z

For reference, more informations about the scheme can be found here: http://wiki.dwarfstd.org/index.php?title=Apple%27s_%22Lazy%22_DWARF_Scheme

The same person has written a SO answer here: https://stackoverflow.com/a/12827463

[dlangBugzillaToGithub]

pro.mathias.lang (@Geod24) commented on 2021-09-28T00:10:43Z

I have a WIP locally so anyone interested, feel free to contact me.
As a reference, this is how LLVM looks up UUID: https://lldb.llvm.org/use/symbols.html

[dlangBugzillaToGithub]

jbc.engelen (@JohanEngelen) commented on 2021-12-30T14:54:55Z

Hi Matthias, 
  Have you considered letting druntime call an external symbolizer like llvm-symbolizer?
https://llvm.org/docs/CommandGuide/llvm-symbolizer.html
llvm-symbolizer will do all the heavy lifting of interpreting dSYM. llvm-symbolizer does not even require the generation of dSYM folder by dsymutil; it's what Address Sanitizer uses, and it works fine for me on macOS.

-Johan

[dlangBugzillaToGithub]

johanengelen commented on 2022-12-17T16:43:46Z

LDC druntime implementation using `atos`: https://github.com/ldc-developers/ldc/pull/4291

[dlangBugzillaToGithub]

sludwig (@s-ludwig) commented on 2024-03-02T11:43:43Z

Just a comment regarding the practicality of the atos approach, which is now also being adopted by DMD - we've had quite bad experiences with this in most situations:

- On some development machines, the call stack terminates after two frames, without any useful information

- In production, it is not possible at all to get useful stack traces from user installations

- Utmost care must be taken to not throw exceptions frequently, because an atos run can easily block the thread for 100ms (or rather not to log the associated stack traces frequently)

All in all, while the earlier situation without line numbers was pretty bad, it has become almost as bad as it can get in our case.