POC: Memory profiler allocation labels

[rudolf]

This is a POC for initial feedback. If we can get alignment within Node.js I could try to contribute the v8 changes upstream.

Summary

Adds the ability to tag sampling heap profiler allocations with string labels that propagate through async context (via CPED). This enables attributing memory usage to specific HTTP routes, tenants, or operations — something no JS runtime currently supports.

V8 changes

• HeapProfileSampleLabelsCallback — embedder callback invoked on sampled allocations to retrieve labels from the current async context
• AllocationProfile::Sample::labels — key-value pairs on each sample (behind V8_HEAP_PROFILER_SAMPLE_LABELS compile flag)

Datadog's Attila Szegedi proposed a similar label mechanism for CPU profiling on v8-dev (July 2025). V8 team (Leszek Swirski) indicated they would review non-invasive patches behind #ifdefs. This PR applies the same approach to heap profiling, which is simpler. Everything runs on the allocation thread with no signal-safety concerns.

Node.js changes:

• v8.withHeapProfileLabels(labels, fn) — runs fn with labels that propagate across await
• v8.setHeapProfileLabels(labels) — sets labels for current async scope (for framework middleware patterns)
• v8.getAllocationProfile() returns samples[].labels and per-label externalBytes (Buffer/ArrayBuffer)
• ProfilingArrayBufferAllocator tracks external allocations per label (single atomic load overhead when disabled)

#62273 landed the SyncHeapProfileHandle API with Symbol.dispose support. The labels API proposed here is complementary, it adds context (which route/tenant) to the samples that SyncHeapProfileHandle already collects. A follow-up could integrate withHeapProfileLabels as a method on the handle.

Motivation

In multi-tenant or multi-route Node.js servers, a memory spike today tells you how much memory grew but not what caused it. Operators resort to code inspection or heap snapshots but these don't scale to collecting data over long timespans for large deployments. With labeled heap/external memory profiling, you can answer "route /api/search accounts for 400MB of the 1.2GB heap" directly from production telemetry (e.g. via OTel).

This mirrors Go's pprof.Labels capability

Overhead

20-run benchmark (two-server realistic HTTP workload):

• Sampling profiler alone: 0.6% (not statistically significant)
• Sampling + labels: 2.2% total (p<0.01)
• When disabled: zero overhead (no code path changes)

Test plan

• V8 cctests for label callback
• JS tests: label propagation across await, concurrent contexts, setHeapProfileLabels, external memory tracking, GC cleanup
• Micro and macro benchmarks in benchmark/v8/ and benchmark/http/

[nodejs-github-bot]

Review requested:

• @nodejs/gyp
• @nodejs/performance
• @nodejs/security-wg
• @nodejs/v8-update

[Qard]

Storing in binding data by the CPED address won't work at all. Because all AsyncLocalStorage contexts are combined into a single AsyncContextFrame map, any changes to any contexts will change what this value is, even if the particular store you are interested in has not changed at all within that map frame.

You would need to have V8 capture the CPED value at the time of the sample and store that on the heap profile itself alongside the samples, then use that actual AsyncContextFrame instance to look up what the corresponding data was in that frame for the label store.

[szegedi]

Very interesting, I had to take a look since I was mentioned in the PR description itself 😄. I generally like the direction this is going in, long term we can probably replace Datadog's heap profiler that directly wraps V8 heap profiler with this and reduce our maintenance surface. This solution indeed looks like it can only be implemented in Node.js itself, and not as an add-on since the v8::ArrayBuffer::Allocator instance is a global Isolate::CreateParams setting so it's controlled by the embedder, that is, Node.js.

[szegedi]

Can't you get the value associated with the AsyncLocalStorage from the AsyncContextFrame here and only store that? Essentially the additional step from BuildSamples. You'll be retaining in memory all ALSes this ACF references as keys, some might have large retained set themselves. If you can safely call GetContinuationPreservedEmbedderData (which creates a v8::Local) I'd think you can also safely get a local to the ALS key from its global, and call v8::Map::Get on context too? (Or direct V8 hashmap reading like I suggested in that other comment in ProfilingArrayBufferAllocator::FindCurrentLabels)

[rudolf]

Great point about memory retention. You're right that storing the entire CPED keeps all ALS stores alive as long as the sample exists, not just the labels store 💣 💥

OrderedHashMap::FindEntry is a great suggestion!

[szegedi]

What's the memory requirement of this AsArray call? It sounds like it'd have to construct a whole new array?
You're lucky that you have a whole embedded copy of V8, so you can use existing internals, something like this roughly sketched might work:

#include "src/objects/js-collection.h"
#include "src/objects/ordered-hash-table.h"

// Given a v8::Local<v8::Map>, get to the internal table:
i::Tagged<i::JSMap> js_map = *Utils::OpenDirectHandle(*frame);
i::Tagged<i::OrderedHashMap> table = i::Cast<i::OrderedHashMap>(js_map->table());

// no-JS lookup in the table:
i::InternalIndex entry = table->FindEntry(isolate, *Utils::OpenDirectHandle(*als_key));
if (entry.is_found()) {
  i::Tagged<i::Object> value = table->ValueAt(entry);
  // go back from Tagged to Local:
  v8::Local<v8::Value> val = Utils::ToLocal(i::direct_handle(value, i_isolate));

  // use val as before...
}

[rudolf]

yeah AsArray() allocates a full JS Array and copies the Map backing store. I moved away from AsArray() to Map::Get() in an unpushed revision, but your OrderedHashMap::FindEntry approach would be even better.

Since we're already modifying V8 source in deps/v8/src/profiler/ and have access to internals from src/ as well, I can adopt this pattern in both locations:

1. SampleObject (allocation time) — extract ALS value, store as Global on Sample
2. ProfilingArrayBufferAllocator::TrackAllocate — same pattern for ArrayBuffer tracking

The read-time callback in GetAllocationProfile then receives the already-extracted flat array, making it trivial (just string conversion).

[szegedi]

So I presume this'll need upstreaming, right?

[rudolf]

yeah, I'm hoping if we can show that Nodejs would definitely use this feature they'd be more open to accepting it

[rudolf]

Very interesting, I had to take a look since I was mentioned in the PR description itself 😄. I generally like the direction this is going in, long term we can probably replace Datadog's heap profiler that directly wraps V8 heap profiler with this and reduce our maintenance surface.

@szegedi Thanks for popping by! Your thread sparked this idea and made me think maybe it's not all that hard (at least for memory profiling, sounds like CPU profiles might be a different beast).

[rudolf]

Great point about memory retention. You're right that storing the entire CPED keeps all ALS stores alive as long as the sample exists, not just the labels store 💣 💥

OrderedHashMap::FindEntry is a great suggestion!

[rudolf]

yeah AsArray() allocates a full JS Array and copies the Map backing store. I moved away from AsArray() to Map::Get() in an unpushed revision, but your OrderedHashMap::FindEntry approach would be even better.

Since we're already modifying V8 source in deps/v8/src/profiler/ and have access to internals from src/ as well, I can adopt this pattern in both locations:

1. SampleObject (allocation time) — extract ALS value, store as Global on Sample
2. ProfilingArrayBufferAllocator::TrackAllocate — same pattern for ArrayBuffer tracking

The read-time callback in GetAllocationProfile then receives the already-extracted flat array, making it trivial (just string conversion).

[rudolf]

yeah, I'm hoping if we can show that Nodejs would definitely use this feature they'd be more open to accepting it

[IlyasShabi]

This means we are storing the label value per sample, even when many samples share the same label set. I think this is likely expensive in memory and in GetAllocationProfile()

IMO it's better to store a small uint32_t label_id = 0 on each sample instead, and keep a shared map for labels. I believe that with this GetAllocationProfile() could resolve labels once per unique context rather than once per sample.

[rudolf]

I agree in principle but wonder if the memory overhead is really an issue. We only retain samples and memory that have survived GC. Starting with per sample overhead: ~120 B base + ~24 B label handle slot = ~144 B. And assuming Poisson sampling with 512kb intervals:

• 200 MB live heap would have ~400 retained samples with 58 KB / 0.029% sample overhead (48 KB base, 10 KB labels)
• 1GB live heap would have ~2000 samples with 288kb sample overhead (240KB base, 48KB labels)

There's additional overhead like the shared ALS array but that's a function of number of unique lables not a per sample overhead so would be present in both designs. Feels worth profiling/benchmarking to make sure my math adds up, but the label_id solution comes at the cost of additional complexity for a small memory gain.

It makes a lot of sense to add a local cache during GetAllocationProfile to avoid the expensive callbacks for labels we have already resolved. I think that can make a noticeable difference to CPU overhead.

[IlyasShabi]

We only retain samples and memory that have survived GC

Probably not if you call it like:

hp->StartSamplingHeapProfiler(
             64, 16,
             static_cast<v8::HeapProfiler::SamplingFlags>(
                   v8::HeapProfiler::kSamplingIncludeObjectsCollectedByMinorGC |
                   v8::HeapProfiler::kSamplingIncludeObjectsCollectedByMajorGC
              )
       );

[rudolf]

@IlyasShabi I'm stuck trying to measure this. The labels work adds bookkeeping that lives in C++ heap so process.memoryUsage().rss and v8.getHeapStatistics().used_heap_size don't directly capture it.

What I've tried so far:

• v8.getHeapStatistics().used_global_handles_size surprisingly doesn't track Sample::label_value Globals
• RSS deltas across 3-iteration matrix runs at 30 s each has a noise band of +/-60 MB

Is there a methodology you'd use to measure this kind of work?

[rudolf]

Did some measurement work on this. With a realistic HTTP workload (~2,500 rps, ~750 MB/s V8 alloc churn), 30 s with includeCollectedObjects: true at64 KB sampling interval, ~300K retained samples: aggregate libc-malloc shows the labels feature adds 6-9 MB on top of ~29 MB of profiler-only bookkeeping. Below 5% of the underlying sampler cost at this workload, scales linearly with sample count.

However, while running the load test I hit a segfault in ProfilingArrayBufferAllocator::TrackFree: V8's ArrayBufferSweeper calls it on a background worker thread, and the per-allocation Global label_value runs ~Global off-thread which trips the node->IsInUse() CHECK in GlobalHandles::NodeSpace::Release.

The fix it I think we need the same machinery as what you suggested for Sample::label_value so will take a stab at that

[IlyasShabi]

We should call ensureHeapProfileLabelsALS() before this line to make sure the ALS store exists and its key is registered before sampling begins.

[szegedi]

@szegedi Thanks for popping by! Your thread sparked this idea and made me think maybe it's not all that hard (at least for memory profiling, sounds like CPU profiles might be a different beast).

Oh yeah, I could probably talk for hours about CPU sample labeling :-)

Reading labels to associate with samples is the tricky part with CPU profiles. Since it happens in a signal handler, we we can't use basically any V8 APIs, as even getting a Local<Value> from GetContinuationPreservedEmbedderData violates signal safety. So in our profiler we end up with code like this as well as a reimplementation of V8 logic for reading hashmaps.

There's also some other fun details, like needing to have a ringbuffer for samples around as you can't allocate memory in the signal handler. And we're also using shared_ptr<Global<Value>> to store the reference to the current label set as we can safely copy a shared_ptr but couldn't be allocating a different global (globals can't be copied either.) FWIW, in our implementation, we aren't converting labels up front, we just take a regular JS object representing them and store a reference to it on sampling, and then only generate labels from its entries when we build the profiles.

Associating labels in that ringbuffer with samples is also somewhat tricky, we do it by correlating monotonic clock values – we invoke v8::base::TimeTicks::Now in the signal handler and store the value in the ring buffer struct and then later correlate them with samples using v8::CpuProfiler::GetSampleTimestamp

As I said, I can probably go on for at least an hour. I might need to write a talk :-)

[Qard]

Reading labels to associate with samples is the tricky part with CPU profiles. Since it happens in a signal handler, we we can't use basically any V8 APIs, as even getting a Local<Value> from GetContinuationPreservedEmbedderData violates signal safety. So in our profiler we end up with code like this as well as a reimplementation of V8 logic for reading hashmaps.

Yeah, this is why I wanted to have V8 itself capture CPED pointer state in each sample internally so that could be read back as actual local values later when you process the profile tree. Never got around to that before I changed teams though.