Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

core: expose gc statistics #6414

Closed
bnoordhuis opened this Issue Oct 25, 2013 · 9 comments

Comments

Projects
None yet
5 participants
Owner

bnoordhuis commented Oct 25, 2013

Tangentially related to #6412. It would benefit profiling and monitoring tools if it was possible to obtain (more) information on garbage collection cycle types and durations.

Suggested solutions: add a V8::GetHeapStatistics() binding, add pre- and post-GC hooks for timing purposes and maybe look into patching V8 to get more detailed GC statistics. For example, you currently can't really differentiate on GC runs in the different spaces.

@ghost ghost assigned bnoordhuis Oct 25, 2013

Member

Qard commented Oct 31, 2013

This'd be pretty handy for checking differences between the before and after state of a GC cycle to see what is and is not getting collected.

Looks good so far! Right now it looks like one needs to call process._heapInfo() in order to get the stats. Would it be possible to register user callbacks pre-/post-GC?

I am similarly +1 on the idea, but I think process is the wrong place for this, I think it's time for us to introduce a module namespace for this and the other pieces we're talking about adding

Member

Qard commented Oct 31, 2013

Agreed, process is getting crowded.
On 30 Oct 2013 17:27, "Timothy J Fontaine" notifications@github.com wrote:

I am similarly +1 on the idea, but I think process is the wrong place for
this, I think it's time for us to introduce a module namespace for this and
the other pieces we're talking about adding


Reply to this email directly or view it on GitHubhttps://github.com/joyent/node/issues/6414#issuecomment-27452029
.

Owner

bnoordhuis commented Oct 31, 2013

I am similarly +1 on the idea, but I think process is the wrong place for this, I think it's time for us to introduce a module namespace for this and the other pieces we're talking about adding

What about require('prof')?

Would it be possible to register user callbacks pre-/post-GC?

Not really, node can't call into JS land from GC hooks. The best we can do is to make the callback as soon as control returns to the event loop but that's not very accurate when the GC was triggered by JS.

I could write something that aggregates GC statistics on behalf of JS land but that will need some fleshing out, API-wise. Suggestions are welcome.

Member

sam-github commented Oct 31, 2013

https://github.com/NodeFly/node-nodefly-gcinfo/blob/master/src/gc.cc#L127

As I read the code, the heap stats are collected after GC and forwarded to main loop, so it's not a callback per-se that occurs after gc, but the collection of heap stats.

Owner

bnoordhuis commented Nov 1, 2013

Right, that's what I mean with aggregating statistics on behalf of JS land. Using uv_queue_work() is kind of wasteful though, that means a round-trip through the thread pool for no reason.

Member

sam-github commented Nov 6, 2013

https://github.com/NodeFly/node-nodefly-gcinfo/blob/master/src/gc.cc#L126, that comment is wrong, isn't it? The v8 docs don't say that, they say no v8 allocation can occur, its a restricted env.

What about uv_send_async() from the callback, with a queue of stats? Async send is usually called from another thread, but it seems it would work from same thread. I wonder if it would even be necessary to protect the async data structure. If the async_cb doesn't do anything that can trigger a gc, it would be safe.

For API, what about an on('gc', function(heapInfo) { but with the addition of 'time of collection' (uv_hrtime)?

Owner

bnoordhuis commented Nov 7, 2013

For API, what about an on('gc', function(heapInfo) { but with the addition of 'time of collection' (uv_hrtime)?

As discussed on IRC: looks like a reasonable suggestion to me but we'll change the prototype of the listener to function(before, after) { /* ... */ }.

@bnoordhuis bnoordhuis removed their assignment May 12, 2015

@bnoordhuis bnoordhuis closed this May 12, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment