[async_hooks] stable API - tracking issue

[AndreasMadsen]

I posted requirements list for getting async_hooks into stable a while ago (nodejs/node#14717 (comment)). This is a very similar list.

Internal Technical

• Clarify breaking changes necessary for better performance (issue: Performance impact of async_hooks benchmarking#181)
  • Remove promise object from PromiseWrap (PR: async_hooks: remove promise object from resource node#23443)
  • Use a "MicrotaskWrap" instead of PromiseWrap (issue: Proposal for reworking promise integration into async_hooks #389)
• Setting up the benchmark suite to be run with and without async_hooks and see that there is only a small measurable difference given our tolerance.
• Deprecate runInAsyncIdScope (issue: Remove async_hooks runInAsyncIdScope API node#14328, PR: async_hooks: deprecate undocumented API node#16972)
• Deprecate setTriggerId (issue: Remove async_hooks setTrigger API node#14238, PR: async_hooks: deprecate undocumented API node#16972)
  • Implement internal replacement that we can optionally expose (PR: Refactor async_hooks initTriggerId node#17273)
• Deprecate Sensitive Embedder API (issue: Remove async_hooks Sensitive/low-level Embedder API node#15572, PR: async_hooks: deprecate undocumented API node#16972)
• Separate internal API from the external API (PR: async_hooks: deprecate undocumented API node#16972).
• Remove deprecated API (PR: async_hooks: remove deprecated API node#17147).
• Opt-out async_hooks state validation in v9.x. (PR: async_hooks: enable runtime checks by default node#16318)
• Make sure all nodecore uses of MakeCallback have an async_context.
• Integrate async_hooks into NAN (issue: Integrate C++ AsyncHooks Embedder API with native abstraction node#13254).
• Integrate async_hooks into n-api (issue: Integrate C++ AsyncHooks Embedder API with native abstraction node#13254).

Internal Non-Technical

• Specify whether a triggerAsyncId change will be a semver-major, semver-minor or semver-patch.
• Identify "tolerance level" of perf impact of async-hooks, especailly regarding promises.
• Is Async Hooks exposing the right logical model? (issue: How to deal with ambgious contexts for promise then callbacks #143)

External Non-Technical

• major APM vendor that uses async_hooks (N/Solid, New Relic, Elastic APM)
• npm modules with at least 5000 DL/month based on async_hooks (https://www.npmjs.com/package/trace & https://www.npmjs.com/package/cls-hooked)

After Stable

• Deprecate MakeCallback without async_context (PR: src: deprecate legacy node::MakeCallback node#18632)

[AndreasMadsen]

/cc @nodejs/async_hooks @nodejs/diagnostics

[AndreasMadsen]

/cc @jasnell who sometimes asks me about this

[refack]

@AndreasMadsen how about adding some external criteria, like TC39 uses or N-API (nodejs/node#14532)

1. major APM vendor that uses async_hooks (N/Solid)
2. npm modules with at least 5000 DL/month based on async_hooks (https://www.npmjs.com/package/trace & https://www.npmjs.com/package/cls-hooked)

[AndreasMadsen]

@refack added your 1. and 2. to the list. I'm not sure what you mean by TC39 uses. Regarding N-API I don't see why that should be a requirement. async_hooks has its own native Embedder API outside N-API, we can mark that as stable without N-API. In fact N-API shouldn't be marked stable before the async_hooks native Embedder API is marked stable since they depend on that. But async_hooks doesn't depend on N-API.

[refack]

Thanks. I just referenced TC39 and N-API as processes that use exit criteria that are independent of the development process, but take into account ecosystem adoption. So I agree that stability of N-API and async_hooks are independent (except for the embedder API)

[vdeturckheim]

FWIW, We plan to start using Async Hooks in Sqreen Agent in the upcomming weeks.

[jkrems]

Link by @watson for elastic: elastic/apm-agent-nodejs#77

[kjin]

PR to add async_hooks in Stackdriver Trace: googleapis/cloud-trace-nodejs#538

[AndreasMadsen]

Updated "Deprecate setTriggerId"

[AndreasMadsen]

Added:

• Clarify breaking changes necessary for better performance (issue: Performance impact of async_hooks benchmarking#181)

[bmeurer]

So based on discussion in the benchmarking WG meeting yesterday I did setup test cases to run the Promise heavy Bluebird and Wikipedia benchmarks (used by the V8 project) with and without async_hooks to answer the first question in this thread, and slow-down is pretty significant even with just an empty init hook.

@mhdawson suggested to kick-off some discussion on the performance via nodejs/benchmarking#188 and bubble up the issue to make sure we consider the performance aspect before async_hooks goes out of EXPERIMENTAL.

[mike-kaufman]

FYI I added an item to list above to identify the perf impact we're willing to tolerate from async-hooks. Current benchmark data cited in nodejs/benchmarking#181 show a ~2x-3x slowdown.

[bnb]

Any thoughts on if this will be coming out of Experimental before v10 launches? Looking at the remaining items I can't quite tell if there are still significant barriers because I'm not familiar enough with the subject 🤔

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.

[kamalmarhubi]

I figure this shouldn't go stale since it's a long-lived tracking issue?

[adamyeats]

Hey team, sorry it has taken me so long to follow up. Last time we spoke I believe you asked me to experiment with the async_hooks API more. We now have an async_hooks implementation in production in our Node.js integration - although it is not much different (if at all) from Cloud Trace's implementation.

I have some thoughts and feedback on the API that it would be great to sync with you all on at some point (if you have time), but it would be good to know if there's anything further you'd like me to do to move this forward?

[mhdawson]

@xadamy maybe you can plan to come ot the next diagnostics meeting and give the team a runthrough of your thoughts.

[mhdawson]

closing in favor of #437