How should profiling interact with worker-threads?

[slonka]

Trying to start the profiler inside worker-thread results in either TypeError: process._startProfilerIdleNotifier is not a function or Module did not self-register error.

I've used this simple example:

const { Worker } = require('worker_threads');

const spin = `
const fs = require('fs');
const pprof = require('pprof');
async function a() {
    const profile = await pprof.time.profile({
        durationMillis: 9000,    // time in milliseconds for which to
                                // collect profile.
    });
    const buf = await pprof.encode(profile);
    fs.writeFile('./wall-worker.pb.gz', buf, (err) => {
        if (err) throw err;
    });
}

a();

function workerSpin() {
    const start = Date.now();
    while (Date.now() - start < 10000);
}
workerSpin();
`;
new Worker(spin, { eval: true });

To try to profile inside a worker.

Full code and sample errors are in my repository: https://github.com/slonka/pprof-nodejs-and-worker-threads

Tested on OSX node 12.6.0 and 11.15.0

[nolanmar511]

Thanks -- I hadn't yet experimented with worker_threads.

Long term, I assume the goal for pprof-nodejs will be to have profiles which are collected outside of worker threads include information about worker threads, so I'm not extremely concerned that profiles cannot be collected within worker threads (except that it does make me think there will be some complications for collecting profiles which include information about worker threads).

Experimenting a bit here, though, the profiles collected right now when there's a worker thread are rather strange.

Heap Allocations, for example, seem to be attributed to call stacks somewhat randomly. The first time I profiled, heap allocations from within the worker were attributed to "keys"; the next time, they were attributed to "Lines":
heap-1.pb.gz
heap-2.pb.gz

The wall profile was more within expected behavior. It was 100% idle, since it only profiled the main thread.
wall.pb.gz

The script I used is attached:
index.js.txt

[nolanmar511]

At this point in time, I don't think this is a feature request that can be prioritized for pprof-nodejs.

Marking as closed (I think a fix here would likely require upstream changes)

[dmitryuk]

Same issue