{{ message }}

# Investigate natvis for improved msvc debugging #36503

Open
opened this issue Sep 15, 2016 · 14 comments
Open

# Investigate natvis for improved msvc debugging#36503

opened this issue Sep 15, 2016 · 14 comments
Labels

### retep998 commented Sep 15, 2016 • edited

 Natvis files are used to provide user friendly visualizations of native objects in the debugger. Currently VS is able to debug Rust programs fairly well, but any sort of type with internal pointers becomes incomprehensible. By providing natvis information for these types, the user can see what is going on. https://msdn.microsoft.com/en-us/library/jj620914 Here is a screenshot of the current situation to make it obvious what needs to be improved: Also, bonus points if you can make the natvis stuff automatically generated from Debug impls. Even if something like VisualRust writes a visualizer plugin thing to do this without natvis, there still needs to be a way to hook into Debug impls to easily get this sort of information. The text was updated successfully, but these errors were encountered:
We are unable to convert the task to an issue at this time. Please try again.

### Boddlnagg commented Sep 15, 2016 • edited

 The debugger will load Natvis files from C++ projects automatically. By default, Natvis files in your project are also inserted into the .pdb file built by the project. This means that if you debug the binary built by this project, the debugger loads the Natvis file from the .pdb even if you do not have the project open. So these natvis files can either be embedded in PDB files or defined per-project, per-user or per-system (see https://msdn.microsoft.com/en-us/library/jj620914.aspx#Anchor_3). Should the Rust stdlib definitions be embedded into each rustc generated PDB file or installed in a central location? The latter option is probably going to be quite difficult given that rustup.rs is to supersede the existing installer and rustup can't do this (the natvis file will depend on the version of the standard library, thus changes with the selected toolchain). For the former option: I think the PDBs are generated by LLVM, so is there any way to tell LLVM to embed the natvis files in there? Writing the natvis files themselves will probably be not too hard. I already got something working for Vec. But there seems to be a problem with the u8 type (also i8), which leads to problems with u8 buffers and especially Vec, which is why I can't get strings to work and which is also why it says for y.data_ptr in the screenshot above. Maybe some problem with the type annotation in the PDB (either Rust's or LLVM's fault)? The other integer types work fine. Anyone who wants to try it: Just add the following as rust.natvis to a VS solution that also contains the exe that you're debugging: {{ length={len} }} len buf.cap len buf.ptr.pointer.__0 Also, bonus points if you can make the natvis stuff automatically generated from Debug impls. This won't work in general, because at least for collections we still want the ability to expand and see the contained items, whereas Debug will just create a flat string. Also the natvis expressions have to be written in a subset of C++ and don't allow function calls.

### retep998 commented Sep 15, 2016

 This won't work in general, because at least for collections we still want the ability to expand and see the contained items, whereas Debug will just create a flat string. Also the natvis expressions have to be written in a subset of C++ and don't allow function calls. The idea is to hook into usage of the Formatter debug helper methods with some really fancy compiler black magic so it generates stuff from a high level. However I can see how it would be difficult, especially for Debug impls with conditional code flow.

### Boddlnagg commented Sep 15, 2016

 Another problem I stumbled upon: I can't figure out a value for the Type attribute to match slices. It should be something like or but that doesn't work, probably because square brackets are not possible in C++ type names 😢

### Boddlnagg commented Sep 16, 2016 • edited

 I investigated a little bit more: There is an option in the VS Debugging settings to enable (verbose) Natvis diagnostic output, and with that enabled I get an "invalid typename" error for slice types (with or without &). I found the following code for type name matching in the MI Debug Engine for VS, which is not the native debugger, but probably very similar in this regard: https://github.com/Microsoft/MIEngine/blob/master/src/MIDebugEngine/Natvis.Impl/NatvisNames.cs So my guess was correct that only C++ types are allowed here. I don't know what's the best thing to do about it. Either get Microsoft to extend the syntax (unfortunately the native debug engine is not open source) or use the native debugger extensibility interface (see https://github.com/Microsoft/ConcordExtensibilitySamples). The latter would definitely require the installation of some extension to VS (such as VisualRust) to use it, but you probably want that anyway to have syntax highlighting and we could then also implement a custom expression evaluator that allows execution of arbitrary Rust expressions while debugging (currently you can use C++ expressions). By the way, there is no understanding of (un-)safety in the debugger obviously, but I think for a custom Rust expression evaluator it would be really interesting to take safety into consideration, so you can't easily violate Rust's safety guarantees while debugging.
mentioned this issue Sep 22, 2016

### Boddlnagg commented Sep 28, 2016

 After manually working around #36646 by patching the PDB, I was able to display the contents of a String correctly: ... using these lines in the natvis file: {vec.buf.ptr.pointer.__0, [vec.len]s8} vec.buf.ptr.pointer.__0, [vec.len]s8 However, I found that not only are type names containing square brackets not accepted, but even &str does not work (VS reports Error: Invalid type name '&str' when loading the natvis file).

### Boddlnagg commented Sep 29, 2016 • edited

 As far as embedding the natvis file inside the PDB is concerned: In recent versions, link.exe has a (mostly) undocumented switch /NATVIS. Pointing that to a natvis file will embed it into the PDB. After doing that, VS (with natvis diagnostics enabled) successfully reports Natvis: Parsing natvis xml file: rust.natvis (from D:\Code\Rust\natvis\test\main.pdb). So we would be able to generate and embed crate-specific natvis definitions, which are probably needed e.g. to display all kinds of enums correctly. However, natvis is really taylored to C++ and the problem with invalid type names mentioned above is still a major blocker for the natvis-only route and might require us to write a custom debugger plugin (which I think should be possible given https://github.com/Microsoft/ConcordExtensibilitySamples), and in that case we might not even need natvis (except maybe as a fallback for people who don't have the debugger plugin installed). On the other hand, we could try to convince the VS guys to accept more typenames ... I think this is really not a technical issue ...
mentioned this issue Oct 7, 2016

### AndrewGaspar commented Feb 13, 2017

 I've spent a little time looking at MSVC target debug-ability this weekend, and I wish I had found this issue sooner! I don't think we should let perfect be the enemy of the good here - we can make most of the types in the built-in crates representable via NatVis. I really think we should go the route of injecting them into the PDBs, and that it would be cool if you could somehow tie .natvis files to crates in the Cargo manifest - when you build, all of your dependent crates' NatVis files get put in your PDB.
mentioned this issue Feb 15, 2017
added a commit to frewsxcv/rust that referenced this issue Feb 15, 2017
Vec, LinkedList, VecDeque, String, and Option NatVis visualizations

I've added some basic [NatVis](https://msdn.microsoft.com/en-us/library/jj620914.aspx) visualizations for core Rust collections and types. This helps address a need filed in issue rust-lang#36503. NatVis visualizations are similar to gdb/lldb pretty printers, but for windbg and the Visual Studio debugger on Windows.

For example, Vec without the supplied NatVis looks like this in windbg using the "dx" command:

0:000> dx some_64_bit_vec
some_64_bit_vec                 [Type: collections::vec::Vec<u64>]
[+0x000] buf              [Type: alloc::raw_vec::RawVec<u64>]
[+0x010] len              : 0x4 [Type: unsigned __int64]


With the NatVis, the elements of the Vec are displayed:

0:000> dx some_64_bit_vec
some_64_bit_vec                 : { size=0x4 } [Type: collections::vec::Vec<u64>]
[<Raw View>]     [Type: collections::vec::Vec<u64>]
[size]           : 0x4 [Type: unsigned __int64]
[capacity]       : 0x4 [Type: unsigned __int64]
[0]              : 0x4 [Type: unsigned __int64]
[1]              : 0x4f [Type: unsigned __int64]
[2]              : 0x1a [Type: unsigned __int64]
[3]              : 0x184 [Type: unsigned __int64]


In fact, the vector can be treated as an array by the NatVis expression evaluator:

0:000> dx some_64_bit_vec[2]
some_64_bit_vec[2] : 0x1a [Type: unsigned __int64]


In general, it works with any NatVis command that understands collections, such as NatVis LINQ expressions:

0:000> dx some_64_bit_vec.Select(x => x * 2)
some_64_bit_vec.Select(x => x * 2)
[0]              : 0x8
[1]              : 0x9e
[2]              : 0x34
[3]              : 0x308


std::string::String is implemented, as well:

0:000> dv
hello_world = "Hello, world!"
empty = ""
new = ""
0:000> dx hello_world
hello_world                 : "Hello, world!" [Type: collections::string::String]
[<Raw View>]     [Type: collections::string::String]
[size]           : 0xd [Type: unsigned __int64]
[capacity]       : 0xd [Type: unsigned __int64]
[0]              : 72 'H' [Type: char]
[1]              : 101 'e' [Type: char]
...
[12]             : 33 '!' [Type: char]
0:000> dx empty
empty                 : "" [Type: collections::string::String]
[<Raw View>]     [Type: collections::string::String]
[size]           : 0x0 [Type: unsigned __int64]
[capacity]       : 0x0 [Type: unsigned __int64]



VecDeque and LinkedList are also implemented.

My biggest concern is the implementation for Option due to the different layouts it can receive based on whether the sentinel value can be embedded with-in the Some value or must be stored separately.

It seems to work, but my testing isn't exhaustive:

0:000> dv
three = { Some 3 }
none = { None }
no_str = { None }
some_str = { Some "Hello!" }
0:000> dx three
three                 : { Some 3 } [Type: core::option::Option<i32>]
[<Raw View>]     [Type: core::option::Option<i32>]
[size]           : 0x1 [Type: ULONG]
[value]          : 3 [Type: int]
[0]              : 3 [Type: int]
0:000> dx none
none                 : { None } [Type: core::option::Option<i32>]
[<Raw View>]     [Type: core::option::Option<i32>]
[size]           : 0x0 [Type: ULONG]
[value]          : 4 [Type: int]
0:000> dx no_str
no_str                 : { None } [Type: core::option::Option<collections::string::String>]
[<Raw View>]     [Type: core::option::Option<collections::string::String>]
[size]           : 0x0 [Type: ULONG]
0:000> dx some_str
some_str                 : { Some "Hello!" } [Type: core::option::Option<collections::string::String>]
[<Raw View>]     [Type: core::option::Option<collections::string::String>]
[size]           : 0x1 [Type: ULONG]
[value]          : 0x4673df710 : "Hello!" [Type: collections::string::String *]
[0]              : "Hello!" [Type: collections::string::String]


For now all of these visualizations work in windbg, but I've only gotten the visualizations in libcore.natvis working in the VS debugger. My priority is windbg, but somebody else may be interested in investigating the issues related to VS.

You can load these visualizations into a windbg sessions using the .nvload command:



There are some issues with the symbols that Rust and LLVM conspire to emit into the PDB that inhibit debugging in windbg generally, and by extension make writing visualizations more difficult. Additionally, there are some bugs in windbg itself that complicate or disable some use of the NatVis visualizations for Rust. Significantly, due to NatVis limitations in windbg around allowable type names, you cannot write a visualization for [T] or str. I'll report separate issues as I isolate them.

In the near term, I hope to fill out these NatVis files with more of Rust's core collections and types. In the long run, I hope that we can ship NatVis files with crates and streamline their deployment when debugging Rust programs on windows.
added a commit to frewsxcv/rust that referenced this issue Feb 16, 2017
Vec, LinkedList, VecDeque, String, and Option NatVis visualizations

I've added some basic [NatVis](https://msdn.microsoft.com/en-us/library/jj620914.aspx) visualizations for core Rust collections and types. This helps address a need filed in issue rust-lang#36503. NatVis visualizations are similar to gdb/lldb pretty printers, but for windbg and the Visual Studio debugger on Windows.

For example, Vec without the supplied NatVis looks like this in windbg using the "dx" command:

0:000> dx some_64_bit_vec
some_64_bit_vec                 [Type: collections::vec::Vec<u64>]
[+0x000] buf              [Type: alloc::raw_vec::RawVec<u64>]
[+0x010] len              : 0x4 [Type: unsigned __int64]


With the NatVis, the elements of the Vec are displayed:

0:000> dx some_64_bit_vec
some_64_bit_vec                 : { size=0x4 } [Type: collections::vec::Vec<u64>]
[<Raw View>]     [Type: collections::vec::Vec<u64>]
[size]           : 0x4 [Type: unsigned __int64]
[capacity]       : 0x4 [Type: unsigned __int64]
[0]              : 0x4 [Type: unsigned __int64]
[1]              : 0x4f [Type: unsigned __int64]
[2]              : 0x1a [Type: unsigned __int64]
[3]              : 0x184 [Type: unsigned __int64]


In fact, the vector can be treated as an array by the NatVis expression evaluator:

0:000> dx some_64_bit_vec[2]
some_64_bit_vec[2] : 0x1a [Type: unsigned __int64]


In general, it works with any NatVis command that understands collections, such as NatVis LINQ expressions:

0:000> dx some_64_bit_vec.Select(x => x * 2)
some_64_bit_vec.Select(x => x * 2)
[0]              : 0x8
[1]              : 0x9e
[2]              : 0x34
[3]              : 0x308


std::string::String is implemented, as well:

0:000> dv
hello_world = "Hello, world!"
empty = ""
new = ""
0:000> dx hello_world
hello_world                 : "Hello, world!" [Type: collections::string::String]
[<Raw View>]     [Type: collections::string::String]
[size]           : 0xd [Type: unsigned __int64]
[capacity]       : 0xd [Type: unsigned __int64]
[0]              : 72 'H' [Type: char]
[1]              : 101 'e' [Type: char]
...
[12]             : 33 '!' [Type: char]
0:000> dx empty
empty                 : "" [Type: collections::string::String]
[<Raw View>]     [Type: collections::string::String]
[size]           : 0x0 [Type: unsigned __int64]
[capacity]       : 0x0 [Type: unsigned __int64]



VecDeque and LinkedList are also implemented.

My biggest concern is the implementation for Option due to the different layouts it can receive based on whether the sentinel value can be embedded with-in the Some value or must be stored separately.

It seems to work, but my testing isn't exhaustive:

0:000> dv
three = { Some 3 }
none = { None }
no_str = { None }
some_str = { Some "Hello!" }
0:000> dx three
three                 : { Some 3 } [Type: core::option::Option<i32>]
[<Raw View>]     [Type: core::option::Option<i32>]
[size]           : 0x1 [Type: ULONG]
[value]          : 3 [Type: int]
[0]              : 3 [Type: int]
0:000> dx none
none                 : { None } [Type: core::option::Option<i32>]
[<Raw View>]     [Type: core::option::Option<i32>]
[size]           : 0x0 [Type: ULONG]
[value]          : 4 [Type: int]
0:000> dx no_str
no_str                 : { None } [Type: core::option::Option<collections::string::String>]
[<Raw View>]     [Type: core::option::Option<collections::string::String>]
[size]           : 0x0 [Type: ULONG]
0:000> dx some_str
some_str                 : { Some "Hello!" } [Type: core::option::Option<collections::string::String>]
[<Raw View>]     [Type: core::option::Option<collections::string::String>]
[size]           : 0x1 [Type: ULONG]
[value]          : 0x4673df710 : "Hello!" [Type: collections::string::String *]
[0]              : "Hello!" [Type: collections::string::String]


For now all of these visualizations work in windbg, but I've only gotten the visualizations in libcore.natvis working in the VS debugger. My priority is windbg, but somebody else may be interested in investigating the issues related to VS.

You can load these visualizations into a windbg sessions using the .nvload command:



There are some issues with the symbols that Rust and LLVM conspire to emit into the PDB that inhibit debugging in windbg generally, and by extension make writing visualizations more difficult. Additionally, there are some bugs in windbg itself that complicate or disable some use of the NatVis visualizations for Rust. Significantly, due to NatVis limitations in windbg around allowable type names, you cannot write a visualization for [T] or str. I'll report separate issues as I isolate them.

In the near term, I hope to fill out these NatVis files with more of Rust's core collections and types. In the long run, I hope that we can ship NatVis files with crates and streamline their deployment when debugging Rust programs on windows.
mentioned this issue Mar 12, 2017

### Mark-Simulacrum commented May 22, 2017

 So a few NatViz visualizations landed in #39843; did that help with this issue? Could someone with access to MSVC check the current status here and report back on what else needs to be added?

### MaulingMonkey commented Jul 11, 2017

This issue mentioned a lot of things in passing. As of rustup-provided:

rustc 1.20.0-nightly (bf0a9e0b4 2017-07-10)
rustc 1.18.0 (03fc9d622 2017-06-06)
Item Status
i8/u8 Fixed in nightly
str.data_ptr Fixed in nightly [1]
Vec/String Implemented in stable
str/[*] Unimplemented [2]
Automatic .natvis selection Unimplemented [3]

[1] This shows unsliced string data - e.g. "asdf\r\n" for "\r\nasdf\r\n".trim() - so we could really use a custom visualizer.

[2] "Bad" characters in typenames still prevent visualizers for these. Mangling the debug typenames to look C++y would be a welcome workaround. A custom debug engine might work OK for windows desktop, but I'm skeptical of how well it'll work for e.g. console platforms which may provide their own DEs. That said, I'm under-educated in this regard.

[3] Rustup now installs .natvis files to e.g. %USERPROFILE%\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\src\etc\natvis\, but these aren't passed to link.exe nor does Visual Rust in any way select them.

mentioned this issue Jul 13, 2017
added a commit that referenced this issue Jul 28, 2017
…erister

Embed MSVC .natvis files into .pdbs and mangle debuginfo for &str, *T, and [T].

No idea if these changes are reasonable - please feel free to suggest changes/rewrites.  And these are some of my first real commits to any rust codebase - *don't* be gentle, and nitpick away, I need to learn! ;)

### Overview
Embedding .natvis files into .pdbs allows MSVC (and potentially other debuggers) to automatically pick up the visualizers without having to do any additional configuration (other than to perhaps add the relevant .pdb paths to symbol search paths.)

The native debug engine for MSVC parses the type names, making various C++ish assumptions about what they mean and adding various limitations to valid type names.  &str cannot be matched against a visualizer, but if we emit str& instead, it'll be recognized as a reference to a str, solving the problem.  [T] is similarly problematic, but emitting slice<T> instead works fine as it looks like a template.  I've been unable to get e.g. slice<u32>& to match visualizers in VS2015u3, so I've gone with str* and slice<u32>* instead.

### Possible Issues
* I'm not sure if slice<T> is a great mangling for [T] or if I should worry about name collisions.
* I'm not sure if linker.rs is the right place to be enumerating natvis files.
* I'm not sure if these type name mangling changes should actually be MSVC specific.  I recall seeing gdb visualizer tests that might be broken if made more general?  I'm hesitant to mess with them without a gdb install.  But perhaps I'm just wracking up technical debt.
Should I try pacman -S mingw-w64-x86_64-gdb and to make things consistent?
* I haven't touched const / mut yet, and I'm worried MSVC might trip up on mut or their placement.
* I may like terse oneliners too much.
* I don't know if there's broader implications for messing with debug type names here.
* I may have been mistaken about bellow test failures being ignorable / unrelated to this changelist.

### Test Failures on x86_64-pc-windows-gnu


---- [debuginfo-gdb] debuginfo-gdb\associated-types.rs stdout ----
thread '[debuginfo-gdb] debuginfo-gdb\associated-types.rs' panicked at 'gdb not available but debuginfo gdb debuginfo test requested', src\tools\compiletest\src\runtest.rs:48:16
note: Run with RUST_BACKTRACE=1 for a backtrace.

[...identical panic causes omitted...]

---- [debuginfo-gdb] debuginfo-gdb\vec.rs stdout ----
thread '[debuginfo-gdb] debuginfo-gdb\vec.rs' panicked at 'gdb not available but debuginfo gdb debuginfo test requested', src\tools\compiletest\src\runtest.rs:48:16


### Relevant Issues
* #40460 Metaissue for Visual Studio debugging Rust
* #36503 Investigate natvis for improved msvc debugging
* PistonDevelopers/VisualRust#160 Debug visualization of Rust data structures

### Pretty Pictures
![Collapsed Watch Window](https://user-images.githubusercontent.com/75894/28180998-e44c7516-67bb-11e7-8b48-d4f9605973ae.png)
![Expanded Watch Window](https://user-images.githubusercontent.com/75894/28181000-e8da252e-67bb-11e7-96b8-d613310c04dc.png)

### abonander commented Apr 10, 2018

 I really would like better enum visualization support. The Visual Studio debugger can tell you the tag of an enum and the variant name, but viewing the contents shows tuple fields for all the variants' contents and you have to guess which field(s) are correct based on the discriminant value and index them visually. It's hell trying to sift through deeply nested enums like those in the compiler frontend.

### lzybkr commented Apr 10, 2018 • edited

 @abonander - indeed, I was writing a natvis entry today and the reuse of field names like __0 and __1 make it difficult to write a correct and safe natvis entry. The best I came up with looks something like: {{ Binary }} {{ Unary }} *(sage_core::constraint::BinaryOp*)(((char*)&RUST$ENUM$DISR) + 1) *(alloc::rc::Rc<sage_core::constraint::Constraint>*)(((char*)&RUST$ENUM$DISR) + sizeof(void*)) *(alloc::rc::Rc<sage_core::constraint::Constraint>*)(((char*)&RUST$ENUM$DISR) + 2*sizeof(void*)) *(sage_core::constraint::UnaryOp*)(((char*)&RUST$ENUM$DISR) + 1) *(alloc::rc::Rc<sage_core::constraint::Constraint>*)(((char*)&RUST$ENUM$DISR) + sizeof(void*)) The real definition is a bit more complicated. Note the explicit/calculated offsets from the enum tag.

### abonander commented Apr 10, 2018

 @lzybkr So it seems like we should be emitting unique field names in the debuginfo for each enum variant... this would definitely make debugging easier in the current situation since you wouldn't have to count up to the right set of fields.

### AndrewGaspar commented Apr 11, 2018

 @abonander If we had this more straightforward method for mapping the discriminator to the debug information for the current value, I think that gets us a lot closer to being able to have an automatic visualizer for all enums. It would be rad, actually, if the compiler just auto-generated natvis files for all enums in builds with debuginfo, and embedded them in the PDB using the link.exe flag.
mentioned this issue Jul 29, 2019
mentioned this issue Nov 21, 2019
added a commit that referenced this issue Nov 23, 2019
…r=michaelwoerister

debuginfo:  Support for std::collections::Hash* in windows debuggers.

Okay, I finally needed to debug code involving a HashMap!  Added support for HashSet s as well.

r? @michaelwoerister

### Local Testing

Verified these are passing locally:
cmd
:: cmd.exe
python x.py test --stage 1 --build x86_64-pc-windows-msvc src/test/debuginfo
python x.py test --stage 1 --build i686-pc-windows-msvc src/test/debuginfo
python x.py test --stage 1 src/tools/tidy

:: MinGW MSYS2
./x.py test --stage 1 --build x86_64-pc-windows-gnu src/test/debuginfo


### Related Issues

* #36503
* #40460
* rust-gamedev/wg#20

### rylev commented Mar 11, 2021

 This issue has overlap with #40460. There has been significant improvement in natvis support since this issue was last commented on. We'll probably want to reconcile the two issues into one tracking issue about debugger support on Windows.