Fix DWARF generation for enums

[tromey]

The DWARF generated for Rust enums was always somewhat unusual.
Rather than using DWARF constructs directly, it would emit magic field
names like "RUST$ENCODED$ENUM$0$Name" and "RUST$ENUM$DISR". Since
PR #45225, though, even this has not worked -- the ad hoc scheme was
not updated to handle the wider variety of niche-filling layout
optimizations now available.

This patch changes the generated DWARF to use the standard tags meant
for this purpose; namely, DW_TAG_variant and DW_TAG_variant_part.

The patch to implement this went in to LLVM 7. In order to work with
older versions of LLVM, and because LLVM doesn't do anything here for
PDB, the existing code is kept as a fallback mode.

Support for this DWARF is in the Rust lldb and in gdb 8.2.

Closes #32920
Closes #32924
Closes #52762
Closes #53153

[rust-highfive]

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

[eddyb]

You can get the size and alignment from niche.value directly.

[eddyb]

Is there no method on Integer to do this conversion?

[tromey]

Not directly (that I could find) but I can use Integer::fit_unsigned.

[eddyb]

Oh, I think you should match on Primitive, convert floats to I32 / I64, and pointers to cx.data_layout().ptr_sized_integer(). That way you don't need to turn an integer back into itself by looking at its size.

[nagisa]

This definitely wants a test or two.

EDIT: though I won’t block this PR if they aren’t easy to write.

[eddyb]

r=me with nits fixed

[eddyb]

r=me with nits fixed

[eddyb]

cc @michaelwoerister @alexcrichton

[nikomatsakis]

r? @eddyb

(I pinged him on Discord as I'm not really that familiar with this code. But I see he already left a review before I came back to re-assign. I agree with @nagisa that we could use more tests here :)

[tromey]

I can write more tests but note that there are existing debuginfo tests that already cover the ordinary cases. The new test is testing the case that could not be made to work previously.

[rust-highfive]

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:50:46] ....................................................................................................
[00:50:50] ....................................................................................................
[00:50:52] ................................i...................................................................
[00:50:55] ....................................................................................................
[00:50:58] .................................................................................iiiiiiiii..........
[00:51:04] ...ii...............................................................................................
[00:51:08] ....................................................................................................
[00:51:11] ..............................................................i.....................................
[00:51:14] ....................................................................................................
---
[00:51:45] ........................................................................................i...........
[00:51:48] ....................................................................................................
[00:51:52] ....................................................................................................
[00:51:54] ....................................................................................................
[00:51:57] .i.ii.ii..ii............................i...........................................................
[00:51:57] test result: ok. 4137 passed; 0 failed; 64 ignored; 0 measured; 0 filtered out
[00:51:57] 
[00:51:57]  finished in 127.128
[00:51:57] travis_fold:end:test_ui_nll
---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-gdb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[00:58:39] 
[00:58:39] running 110 tests
[00:58:51] iiii.....F.i..i........i..i.i...F.....F..Fi..........iiii...........i....i..........ii.i.i...F...ii.
ne at:
[00:58:51] <http://www.gnu.org/software/gdb/documentation/>.
[00:58:51] For help, type "help".
[00:58:51] Type "apropos word" to search for commands related to "word".
[00:58:51] Breakpoint 1 at 0xb12: file /checkout/src/test/debuginfo/borrowed-enum.rs, line 81.
[00:58:51] [Thread debugging using libthread_db enabled]
[00:58:51] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[00:58:51] 
[00:58:51] Breakpoint 1, borrowed_enum::main::hfdf2cbe4fde99803 () at /checkout/src/test/debuginfo/borrowed-enum.rs:81
[00:58:51] 81     zzz(); // #break
[00:58:51] $1 = {TheA = {RUST$ENUM$DISR = TheA, x = 0, y = 8970181431921507452}, TheB = {RUST$ENUM$DISR = TheA, __0 = 8970181431921507452, __1 = 32767, __2 = 0}}
[00:58:51] $2 = {TheA = {RUST$ENUM$DISR = TheB, x = 140733479719185, y = 0}, TheB = {RUST$ENUM$DISR = TheB, __0 = 0, __1 = 286331153, __2 = 286331153}}
[00:58:51] $3 = {TheOnlyCase = {__0 = 4820353753753434}}
[00:58:51] A debugging session is active.
[00:58:51] 
[00:58:51]  Inferior 1 [process 14603] will be killed.
[00:58:51] 
[00:58:51] Quit anyway? (y or n) [answered Y; input not from terminal]
[00:58:51] ------------------------------------------
[00:58:51] stderr:
[00:58:51] ------------------------------------------
[00:58:51] 
---
[00:58:51] 
[00:58:51] ---- [debuginfo-gdb] debuginfo/generic-enum-with:58:51] 
[00:58:51] thread '[debuginfo-gdb] debuginfo/generic-enum-with-different-disr-sizes.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:3196:9
[00:58:51] 
[00:58:51] ---- [debuginfo-gdb] debuginfo/generic-struct-style-enum.rs stdout ----
[00:58:51] NOTE: compiletest thinks it is using GDB without native rust support
[00:58:51] NOTE: compiletest thinks it is using GDB version 7011001
[00:58:51] 
[00:58:51] error: line not found in debugger output: $1 = {{RUST$ENUM$DISR = Case1, a = 0, b = 31868, c = 31868, d = 31868, e = 31868}, {RUST$ENUM$DISR = Case1, [...]}, {RUST$ENUM$DISR = Case1, [...]}}
[00:58:51] status: exit code: 0
[00:58:51] command: "/usr/bin/gdb" "-quiet" "-batch" "-nx" "-command=/checkout/obj/build/x86_64-unknown-linux-gnu/test/debuginfo/generic-struct-style-enum/generic-struct-style-enum.debugger.script"
[00:58:51] ------------------------------------------
[00:58:51] GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
[00:58:51] Copyright (C) 2016 Free Software Foundation, Inc.
[00:58:51] Copyright (C) 2016 Free Software Foundation, Inc.
[00:58:51] License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
[00:58:51] This is free software: you are free to change and redistribute it.
[00:58:51] There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
[00:58:51] and "show warranty" for details.
[00:58:51] This GDB was configured as "x86_64-linux-gnu".
[00:58:51] Type "show configuration" for configuration details.
[00:58:51] For bug reporting instructions, please see:
[00:58:51] <http://www.gnu.org/software/gdb/bugs/>.
[00:58:51] Find the GDB manual and other documentation resources online at:
[00:58:51] <http://www.gnu.org/software/gdb/documentation/>.
[00:58:51] For help, type "help".
[00:58:51] Type "apropos word" to search for commands related to "word".
[00:58:51] Breakpoint 1 at 0x9f7: file /checkout/src/test/debuginfo/generic-struct-style-enum.rs, line 86.
[00:58:51] [Thread debugging using libthread_db enabled]
[00:58:51] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[00:58:51] 
[00:58:51] Breakpoint 1, generic_struct_style_enum::main::h6baa2d814575bd09 () at /checkout/src/test/debuginfo/generic-struct-style-enum.rs:86
[00:58:51] 86     zzz(); // #break
[00:58:51] $1 = {Case1 = {RUST$ENUM$DISR = Case1, a = 0, b = 31868, c = 31868, d = 31868, e = 31868}, Case2 = {RUST$ENUM$DISR = Case1, a = 0, b = 2088533116, c = 4137712764}, Case3 = {RUST$ENUM$DISR = Case1, a = 140737331100796, b = 0}}
[00:58:51] $2 = {Case1 = {RUST$ENUM$DISR = Case2, a = 0, b = -2133, c = 4369, d = 4369, e = 4369}, Case2 = {RUST$ENUM$DISR = Case2, a = 0, b = 286331153, c = 286331153}, Case3 = {RUST$ENUM$DISR = Case2, a = 140733479719185, b = 0}}
[00:58:51] $3 = {Case1 = {RUST$ENUM$DISR = Case3, a = 6438275382588823897, b = 63405, c = 32767, d = 0, e = 0}, Case2 = {RUST$ENUM$DISR = Case3, a = 6438275382588823897, b = 32767, c = 0}, Case3 = {RUST$ENUM$DISR = Case3, a = 0, b = 6438275382588823897}}
[00:58:51] $4 = {TheOnlyCase = {a = -1}}
[00:58:51] A debugging session is active.
[00:58:51] 
[00:58:51]  Inferior 1 [process 15075] will be killed.
[00:58:51] 
[00:58:51] Quit anyway? (y or n) [answered Y; input not from terminal]
[00:58:51] 
[00:58:51] ------------------------------ug reporting instructions, please see:
[00:58:51] <http://www.gnu.org/software/gdb/bugs/>.
[00:58:51] Find the GDB manual and other documentation resources online at:
[00:58:51] <http://www.gnu.org/software/gdb/documentation/>.
[00:58:51] For help, type "help".
[00:58:51] Type "apropos word" to search for commands related to "word".
[00:58:51] Breakpoint 1 at 0x9f8: file /checkout/src/test/debuginfo/generic-tuple-style-enum.rs, line 104.
[00:58:51] [Thread debugging using libthread_db enabled]
[00:58:51] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[00:58:51] 
[00:58:51] Breakpoint 1, generic_tuple_style_enum::main::h0ff66a8041e01145 () at /checkout/src/test/debuginfo/generic-tuple-style-enum.rs:104
[00:58:51] 104     zzz(); // #break
[00:58:51] $1 = {Case1 = {RUST$ENUM$DISR = Case1, __0 = 0, __1 = 31868, __2 = 31868, __3 = 31868, __4 = 31868}, Case2 = {RUST$ENUM$DISR = Case1, __0 = 0, __1 = 2088533116, __2 = 4158487676}, Case3 = {RUST$ENUM$DISR = Case1, __0 = 140737351875708, __1 = 0}}
[00:58:51] $2 = {Case1 = {RUST$ENUM$DISR = Case2, __0 = 0, __1 = -2400, __2 = 4369, __3 = 4369, __4 = 4369}, Case2 = {RUST$ENUM$DISR = Case2, __0 = 0, __1 = 286331153, __2 = 286331153}, Case3 = {RUST$ENUM$DISR = Case2, __0 = 140733479719185, __1 = 0}}
[00:58:51] $3 = {Case1 = {RUST$ENUM$DISR = Case3, __0 = 6438275382588823897, __1 = -2131, __2 = 32767, __3 = 0, __4 = 0}, Case2 = {RUST$ENUM$DISR = Case3, __0 = 6438275382588823897, __1 = 32767, __2 = 0}, Case3 = {RUST$ENUM$DISR = Case3, __0 = 0, __1 = 6438275382588823897}}
[00:58:51] $4 = {TheOnlyCase = {__0 = -1}}
[00:58:51] A debugging session is ax-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-5.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "5.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[00:58:51] 
[00:58:51] 
[00:58:51] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[00:58:51] Build completed unsuccessfully in 0:12:16
[00:58:51] Build completed unsuccessfully in 0:12:16
[00:58:51] make: *** [check] Error 1
[00:58:51] Makefile:58: recipe for target 'check' failed

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:06cbc26b
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_time:end:0e4653d4:start=1536264235832955119,finish=1536264235841468195,duration=8513076
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:11ebf6c4
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/core

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[tromey]

Still investigating that failure. I'd tried things out with llvm 6 but not 5. I somewhat recall having to make a change here for 6; I will try all 3 ways again but if something has to regress, I think we should let it be 5.

[tromey]

I built rust against LLVM 5, 6, and rust-llvm; and then I tested each one against gdb 7.11, 8.1, and 8.2.

This found a crash in 8.2, filed: https://sourceware.org/bugzilla/show_bug.cgi?id=23626. I'll fix this before 8.2.1.

I couldn't reproduce the bot's failure; but the bot is using 7.11, and my feeling is that a minor output regression there is nothing to worry about. So, if it persists and there are no objections, I'll just bump the minimum gdb version for the test.

[tromey]

I couldn't reproduce the bot's failure

I rebased & tried again and now it failed.

[tromey]

I rebased & tried again and now it failed.

At some point I thought I needed to give the variants names, but reverting this and re-testing didn't show any problems.

[tromey]

@eddyb can you re-review? I don't think I have sufficient permissions to mark it. Thanks.

[eddyb]

@tromey Is the compiler-builtins submodule change intentional?

[eddyb]

Hmm, I think if you do this then we're "insured" from weird 128-bit bugs:

let niche = (i as u128)
    .wrapping_sub(*niche_variants.start() as u128)
    .wrapping_add(niche_start);
assert_eq!(niche as u64 as u128, niche);
Some(niche as u64)

Note that other than the assert, you also had an overflow bug: if niche_variants.start() is larger than i, subtracting u64s and casting the "negative" result to u128 puts you close to 1 << 64 (because it zero-extends), instead of being a "negative" 128-bit integer.

[eddyb]

Hmm, do all variants use these size/alignment values? Because each variant has its own "sub-layout" that "fits" inside the enum's layout, and it might make more sense to use those individual ones. Also, instead of UNKNOWN_LINE_NUMBER, you can get each variant's declaration span.

[tromey]

This part is just a bit of DWARF formalism. This is emitting a DW_TAG_variant_part, which according to DWARF is the part of an enclosing structure that can vary. For Rust, the entire thing can vary, so we emit a structure wrapping a variant part, which wraps the variants. Since there can only be one variant part, it makes sense for it to fill the enclosing structure. And, the location of this thing is unimportant as it is basically entirely artificial -- a synthetic construct only necessary because this is how DWARF mandates it.

[eddyb]

Ah, so it's not per-variant?

[tromey]

Nope.

[tromey]

@tromey Is the compiler-builtins submodule change intentional?

Nope.

[tromey]

@bors r+

[bors]

📌 Commit 98b2688 has been approved by tromey

[bors]

⌛️ Testing commit 98b2688 with merge 0db7abe...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: tromey
Pushing 0db7abe to master...

[michaelwoerister]

🎉 🎉 🎉 !