Self hosted x86_64 backend appears to emit bad debug info on Linux

[whereswaldon]

Zig Version

0.16.0-dev.11+01b502386

Steps to Reproduce and Observed Behavior

Make this simple zig project on x86_64 Linux:

//build.zig
const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const mod = b.addModule("zig_repro", .{
        .root_source_file = b.path("src/root.zig"),
        .target = target,
    });

    const mod_tests = b.addTest(.{
        .root_module = mod,
    });

    const run_mod_tests = b.addRunArtifact(mod_tests);

    const test_step = b.step("test", "Run tests");
    test_step.dependOn(&run_mod_tests.step);
}

//src/root.zig
//! By convention, root.zig is the root source file when making a library.
const std = @import("std");

pub fn bufferedPrint() !void {
    // Stdout is for the actual output of your application, for example if you
    // are implementing gzip, then only the compressed bytes should be sent to
    // stdout, not any debugging messages.
    var stdout_buffer: [1024]u8 = undefined;
    var stdout_writer = std.fs.File.stdout().writer(&stdout_buffer);
    const stdout = &stdout_writer.interface;

    try stdout.print("Run `zig build test` to run the tests.\n", .{});

    try stdout.flush(); // Don't forget to flush!
}

pub fn add(a: i32, b: i32) i32 {
    return a + b;
}

test "basic add functionality" {
    try std.testing.expect(add(3, 7) == 10);
}

Run a zig build test to generate a test executable in ./zig-cache. Find that executable in something like ./.zig-cache/o/4b05f9f63a9dee57fa29fdff781ea789/test.

Try to debug the tests with gdb or lldb. Something like:

$ lldb .zig-cache/o/4b05f9f63a9dee57fa29fdff781ea789/test 
(lldb) target create ".zig-cache/o/4b05f9f63a9dee57fa29fdff781ea789/test"
Current executable set to '/home/chris/Code/tapestry-research/zig-repro/.zig-cache/o/4b05f9f63a9dee57fa29fdff781ea789/test' (x86_64).
(lldb) b add
Breakpoint 1: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
(lldb) 

Now add .use_llvm=true to the mod_tests declaration in build.zig like this:

    const mod_tests = b.addTest(.{
        .root_module = mod,
        .use_llvm = true,
    });

Rerun zig build test to regenerate a test executable and try to debug the new executable (at its new path):

$ lldb .zig-cache/o/8f960bbd12346026c6882d2732508bad/test
(lldb) target create ".zig-cache/o/8f960bbd12346026c6882d2732508bad/test"
Current executable set to '/home/chris/Code/tapestry-research/zig-repro/.zig-cache/o/8f960bbd12346026c6882d2732508bad/test' (x86_64).
(lldb) b add
Breakpoint 1: where = test`root.add + 12 at root.zig:17:32, address = 0x0000000001035d4c
(lldb)

Expected Behavior

I expected the debugger to be able to set breakpoints in code generated by the new x86_64 backend. I couldn't find any existing issues that seemed to cover this. I apologize if I missed one. The closest related thing seems like #21719, but that's specific to macOS, whereas I'm on Linux.

[jacobly0]

No debugger has added support for Zig debug info.

[whereswaldon]

@jacobly0 Thanks for the response, but I don't think I understand. Are you saying that Zig's self-hosted backend uses an entirely different debug info format? Why is that? Is there somewhere I can learn more about this?

[andrewrk]

It's DWARF 5, with a few extended opcodes used here and there.

Indeed, it's an opportunity to attract more debugger implementations by being more thorough with the documentation.

[jacobly0]

External discussion reproduced here, and I can also answer any other questions.

DW_LNE_ZIG_set_decl has a single operand of DW_FORM_ref_addr and it sets the current addr, line, column, and file to that of the referenced DIE. These should only be emitted to incremental binaries, and not to non-incremental binaries created by the self-hosted backend.
The reference impl is here
Just to make sure I didn't forget something that it does.
After the DW_LNE_ZIG_set_decl, all line number changes are relative, and so when that decl gets moved to a different line number (such as when code is inserted earlier in the file) during an incremental update, but otherwise consists of the same source code, only the line number in one DIE is updated.

[whereswaldon]

Okay, let me see if I understand correctly:

In general Zig's self-hosted backend uses DWARF 5, which none of the debuggers support.

In addition, Zig's incremental builds are using an extended opcode that makes it possible/easier to patch incremental binaries without breaking their debug info. That's not relevant for non-incremental builds, which are just blocked by lack of DWARF 5 support in general.

Is that reasonably accurate?

[jacobly0]

The PoC linked above demonstrates (through tests run by our CI) that it is possible to set breakpoints using the debug info that is currently being emitted. While there are more extensions that are also used in non-incremental binaries, I suspect that non-functional breakpoints are not even caused by unimplemented extensions, but just from not implementing logic for handling Zig scoping rules.

[whereswaldon]

I suspect that non-functional breakpoints are not even caused by unimplemented extensions, but just from not implementing logic for handling Zig scoping rules.

Why is special scoping logic in the debugger necessary now when it wasn't needed when using the LLVM backend? Sorry if this should be obvious. 😅

[jacobly0]

LLVM doesn't support emitting Zig debug info either, so it is currently implementing "C++" debug info which is not an acceptable long-term solution.

[whereswaldon]

Ahh, so we're sorta approximating accurate debug info by pretending to be C++ in the DWARF on LLVM, but we are emitting accurate data on the self-hosted backend. I think I understand. Thanks for taking the time to explain!