Can't trace all GC allocations with ETW

[vvuk]

While working on coreclr ETW events integration with the samply profiler, I realized that there's no per-object allocation event. GCAllocationTick only fires when the threshhold goes over, which is a const set at 100kb. However, the CORProfiler interface does have this -- COR_PRF_MONITOR/ENABLE_OBJECT_ALLOCATED and friends. These trigger a callback via the profiler interface.

Is there interest in adding this? It would be expensive, so only if a specific new provider was enabled, similar to the rundown provider? (Or does this already exist and I'm missing it?)

[dotnet-policy-service]

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

[tommcdon]

@noahfalk

[noahfalk]

It would be expensive, so only if a specific new provider was enabled, similar to the rundown provider? (Or does this already exist and I'm missing it?)

Agreed it would have high overhead. As long as only the scenarios which opt-in to that level of tracing pay the perf cost for it I'd have no objection though. And you are correct its not there - you didn't miss anything :)

Is there interest in adding this?

I'm not aware of any requests from specific profiler authors to add it aside from yours. Ignoring development costs I'd say it sounds like a handy feature. Bringing cost back into the picture I'd say its not a priority to implement based on what I know so far.

However, the CORProfiler interface does have this -- COR_PRF_MONITOR/ENABLE_OBJECT_ALLOCATED and friends. These trigger a callback via the profiler interface.

Yep! Currently this is the only approach I'm aware of to do what you are interested in.
One limitation to be aware of - the profiler API requires that the event is requested at process startup. This is so that the runtime can tell the JIT not to use our fast object allocation assembly helpers and instead we always fallback to the slow paths that are capable of logging events. Thats also the issue that makes capturing allocation data from ETW/EventPipe/etc non-trivial. The event tracing tools are allowed to start tracing after startup has already happened.

[vvuk]

This is so that the runtime can tell the JIT not to use our fast object allocation assembly helpers and instead we always fallback to the slow paths that are capable of logging events. Thats also the issue that makes capturing allocation data from ETW/EventPipe/etc non-trivial. The event tracing tools are allowed to start tracing after startup has already happened

Yeah, I noticed this in Unity's profiler hookups. I haven't looked at the code yet, but I'd likely implement the same restriction in the first version -- the provider would need to be enabled at process startup.

I'll take a look at doing this. I'm trying to push people to do profiling with ETW (perfview, samply, wpa, etc.) and given how semi-obsessed we are about GC allocs, this will be pretty useful. Our own profiler isn't a sampling one, so while you get all the GC allocs and we can even capture stacks, it's often not super useful unless you already know exactly what you're looking for.

[vvuk]

Hmm, wait a sec -- isn't GCSampledObjectAllocationHigh this? ref https://medium.com/criteo-engineering/build-your-own-net-memory-profiler-in-c-allocations-1-2-9c9f0c86cefd and looking at the code, it looks like this is fired inside SendObjectAllocatedEvent

[noahfalk]

Apologies, my mistake! I knew we had these high/low sampling events but I didn't realize there was a conditional written in the code that when you specify both of the keywords together instead of sampling it returns all allocations instead.

runtime/src/coreclr/vm/eventtrace.cpp

Line 1025 in 71f8fb6

// If both the high and low verbosity keywords are enabled, log all allocations.

Indeed this looks like what you wanted was hiding in there and I never realized it. Sorry for pointing in the wrong direction!

[vvuk]

Closing, as this exists!