Add .NET support

[benaadams]

There is already a cross platform (Windows, macOS, Linux) sampling profiler for .NET as part of the platform which can be invoked with dotnet trace collect (instructions)

Would be great it this could be adapted or utilised to serve as an agent for pyroscope.

Have also raised an issue in the .NET diagnostics repo regarding this dotnet/diagnostics#2068

[juniortads]

I would like to hear about the next steps on this issue :)

[petethepig]

Did some research, here is a rough TODO list:

• (optional) add windows support, moved this to a separate issue (Windows support #166)
• implement a parser for dotnet profiler data format (see comment below and Add support for speedscope format #172 for more information)
• as for the integration itself, we could add a new dotnetspy and copy the architecture after ebpfspy integration we already have. That one starts a script (it's profile.py in ebpfspy case, for dotnetspy it will be dotnet trace collect), parses the profiling data from folded format to internal pyroscope format and the rest is handled by pyroscope agent code
• add an example to examples/dotnet
• update docs

[petethepig]

So it looks like default dotnet format (nettrace) is hard to parse. I think for now we could use dotnet trace with --format speedscope argument, speedscope format looks much simpler. Plus if we added support for speedscope format it would also make it easier to create other integrations in the future.

I made a separate issue for that #172

[benaadams]

Also does --format Chromium which is the one used by browser tools

[kolesnikovae]

There is one more option to get data from .Net runtime - to use Diagnostics IPC directly, bypassing dotnet-trace tool. This allows us to have more accurate measurements, eliminate some overhead caused by additional serialisation, and be independent of dotnet-trace (which is not installed by default). I’ve already implemented a basic IPC client that can access raw EventPipe stream and now I’m working on the decoder.

What do you think guys @benaadams, @petethepig?

[benaadams]

There is one more option to get data from .Net runtime - to use Diagnostics IPC directly, bypassing dotnet-trace tool. This allows us to have more accurate measurements, eliminate some overhead caused by additional serialisation, and be independent of dotnet-trace (which is not installed by default). I’ve already implemented a basic IPC client that can access raw EventPipe stream and now I’m working on the decoder.

@tommcdon @josalem would this be a good approach?

[josalem]

The CLI tools are (mostly) thin wrappers around the Microsoft.Diagnostics.NetCore.Client NuGet package. That has everything you need to start an EventPipe session (or other diagnostic tasks) without having to spin your own Diagnostics IPC Protocol implementation. As for parsing the event stream, you can use EventPipeEventSource from the Microsoft.Diagnostics.TraceEvent package. That has a callback mechanism for inspecting events live or parsing a stream of events that's been written to disk (in nettrace format). The TraceEvent package also has logic for examining stack samples.

Those components are all written in C#, so if you want to write the agent in something other than that, you will have to spin your own implementation for now.

I'd be happy to chat some more about your use case if you have any questions 😄.

[petethepig]

@josalem Thanks for the pointers! This is very useful.

@kolesnikovae @benaadams I think we're gonna go with the Diagnostics IPC route for the reasons @kolesnikovae mentioned:

• eliminate some overhead caused by additional serialization
• allows us to have more accurate measurements
• no dependency on dotnet-trace