Support watch window extensions from EXE directly?

[cmuratori]

So, I don't know what the exact plan is for extending the watch window, but I wanted to throw this out there for consideration:

Among the many problems with Visual Studio is that it is epic-ly cumbersome to add custom types to the watch window. If you really want to spend some time and energy extending it, you can, but it's nothing something you're just going to do in 5 seconds to inspect some custom tree structure in your game that you're only just debugging for a few hours that day, etc.

But I think it could be made this simple in Remedy with some planning.

My idea is:

1. The plug-in API for Remedy watches will want to be refined and simple, hopefully something that you can just #include as a single .h file and it works. We can work on this to make sure it is as simple as possible and easy to code new watches, but let's assume it's possible.

2. You can build a plug-in DLL if you want, which uses this API, and that's fine for things like "generic bitmap viewer" or whatnot, but...

3. ... Remedy also looks for extensions in your EXE to see if it exports any watch window extensions.

This last one is the crucial one. If anyone making a game can just quickly, right above the thing they're debugging, do something like:

REMEDY_WATCH(my_struct_type)
{
    // ... textification and/or drawing code goes here ...
}

This would allow for a really dramatic improvement in everyone's ability to quickly create custom displays for what they are debugging at the moment.

I'm not sure the best way to expose this at the Windows loading level (eg., normally you would have to enumerate the symbols exported from the EXE, but since Remedy is a debugger and has the PDB, I don't know if perhaps that is an easier way for it to know these things?) But I think it's plausible.

- Casey

[Manuzor]

I like this idea a lot. I'm a fan of stb-style single-header libraries and always wanted to define debugger visualizations (natvis as its called in VS) right in the header file so I can truly just distribute one file. Right now, I'm copy pasting the contents of the .natvis file into the header as a comment.

Here are some thoughts I'd like to share, hoping that it's useful:

• The API should translate easily to non-c/non-preprocessed languages like Jai or zig, i.e. pure C89 with minimal preprocessor usage
• How would libraries specify debugger visualizations? I guess by bracketing their REMEDY_WATCH declaration with an #ifdef. Or by putting their watch declarations in a dedicated header that the user of the library has to include manually.
• A declarative approach (like natvis) could parse markup within comments in source files (they would have to be present for this to work, though). Something like // clang-format off.
• Visual Studio automatically reloads a natvis file if it changed on disk. This is super helpful when tweaking a natvis file
• VS natvis files are already in use by many, it would be nice if there was a way to make remedybg understand them (plugin?), or have a converter to put it in a format that remedybg understands (e.g. generated header you can include). I'm sure some people or there would be glad if they didn't have to maintain multiple versions of their debugger visualizations.

It used to be a pain to get custom types in the VS watch window, especially if you wanted to share your settings. But with natvis it's gotten a lot easier. You define an XML file like this (excerpt from imgui.natvis):

<?xml version="1.0" encoding="utf-8"?>

<!-- natvis file for Visual Studio debugger (you can include this in a project file, or install in visual studio folder) -->

<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">

<Type Name="ImVector&lt;*&gt;">
  <DisplayString>{{Size={Size} Capacity={Capacity}}}</DisplayString>
  <Expand>
    <ArrayItems>
      <Size>Size</Size>
      <ValuePointer>Data</ValuePointer>
    </ArrayItems>
  </Expand>
</Type>

</AutoVisualizer>

It's not ideal but it's not that bad either.

[MazingR]

Hi, I'm tying to use this tool alongside JetBrains Rider (which has an atrocious GDB debugger) to work on UnrealEngine code.
Unfortunatly, its basicaly unusable without some sort of .natvis file support (Unreal has more than a 100 types visualizers defined).

I'd be very interested by any advances on that part because this project looks promising. Very impressed by the overall velocity.

• MGZ

[simonvanbernem]

I like Casey's idea very much!

Natvis may be an improvement for Visual Studio, but it's still terrible for what it does: Visualizing something just a little bit sophisticated, like a pool allocator that has an occupancy bitmask/bool array is a trial and error parade if you haven't done it before. Error reporting and documentation is lackluster, and while doing it all i can think of is "this could have been 5 minutes of coding instead of an hour of google". I personally think it would be better if remedy didn't adopt this solution, it's just not a good one.

Matching the functionality of natvis could be done with a fairly small API I believe, but if we were willing to go one step further, much more powerfull visualizations would be possible: I'm currently working with geometric algorithms, and it would be super helpful if I could create my own format specifiers that take arrays of vertices and plot them as points/lines/a polygon right there in a watch window.

Maybe the API could expose the imgui context for this? This is quite a messy solution (and maybe you don't want to make it THAT easy for the visualizers to crash the debugger), but there are tons of great DearImgui widgets out there that people could use for their custom stuff. Or maybe the debugger just provides functions to draw primitives and query inputs and so on.

In any case, I hope a custom visualizer solution will make it into the debugger sooner or later! Just being able to tell it that my string type has ptr+length would be much appreciated.

[Pere001]

This would be cool but I think Casey's idea would cause more friction than need be for most situations. Just have a window in RemedyBG to configure struct visualization. This way it's adaptive (you don't have to think ahead or recompile) and it's simpler (you don't have to program, have more files, etc.) and simpler to learn (because it's more interactive). Plus my idea would be trivial to implement.

Make a new window where the user can assign and de/activate expressions to struct types. Then, when watching a variable/expression of that type in the watch window or the editor, the struct will be visualized based on the expression(s) defined for that type. Since Remedy is presumably going to have visualizations that cover most use cases, this will also cover most use cases.

I think it would be most practical to have both: per-type expressions editable from RemedyBG will be convenient for 90% of situations, and a programming interface will take care of the rest. It's easy to see how both could seamlessly coexist.

[cmuratori]

I don't disagree. While for me it's not really much different, that's because I keep compile times extremely low. I often forget that a lot of projects take 30 minutes to compile or whatnot, so their turn-around times even for localized changes may be tens of seconds making an embedded watch system slow to iterate on...

- Casey

[forrestthewoods]

3. ... Remedy also looks for extensions in your EXE to see if it exports any watch window extensions.

Sounds awesome. Is there a reasonable way to do this? My cursory glances on the internet lead me to believe there's no simple way to load an exe like it's a DLL via LoadLibrary. Which is a bummer. If a game module was a DLL then it'd be easy to LoadLibrary and scrape exported functions for REMEDYBG_WATCH_typename or some such.

It can maybe be done with some pretty gross hacks. But it also requires the user to change their compilation settings which I don't think is particularly reasonable. https://www.codeproject.com/Articles/1045674/Load-EXE-as-DLL-Mission-Possible

Instead of having to write and deploy a custom DLL would it be reasonable to define a Lua API (or similar)? IIRC Natvis bakes some pretty hard assumption into how containers are defined and pointers are followed. But I suspect 99% of custom data types don't need elaborate custom visualization code.

I suppose the options are:

1. do nothing
2. support natvis (limited schema)
3. custom schema
4. scripting language support
5. native plugin
6. native plugin w/ automagic loading from exe

I feel like 4 could be a pretty good point in terms of low-cost / high-value.

[cmuratori]

LoadLibrary and GetProcAddress work on both EXEs as DLLs, you do not have to do anything special.

- Casey

[forrestthewoods]

Oh! I misinterpreted things. That's great.

Yes, it looks like you can LoadLibrary and GetProcAddress on an exe. But when doing this with an exe it won't initialize the CRT, globals, or the import table. Which is... maybe fine? The link has a clean enough solution to the import table (maybe). I don't like the gross work around necessary to init globals and CRT though. It requires modifying the user main which isn't cool.

It feels like there should be a function that initializes CRT but doesn't call WinMain / DllMain. But I haven't found it yet. I'm sure there's some hackery that can be done.

I did a basic test and yeah you can totally load an exe, call GetProcAddress, and call the function. So long as RemedyBG passes everything the exe function could possibly need that should do it? This path will also "just work" if the game code being debugged was in a DLL.

If folks wanted to get real fancy then RemedyBG could pass a full ImGui interface potentially with extensions like ImPlot. That's probably overkill. But hey maybe not.

[danielmaciel]

REMEDY_WATCH(my_struct_type)
{
    // ... textification and/or drawing code goes here ...
}

Dude that macro thing would be pretty great. Could use natvis support as well since there are some files floating around the web to reuse.