feat: pre-register partial transactions in AWS Lambda

[trentm]

When used with the Lambda extension >=v1.4.0, this results in
transactions being reported for Lambda timeout, uncaughtException,
and unhandledRejection.

Closes: #3136
Closes: #2379

checklist

• depends on v11.4.0; support for pre-registering partial transactions for AWS Lambda apm-nodejs-http-client#199 and updating the package.json dep to a release of the http-client

follow-up work

• Drop AgentMock in tests and re-add support for contextManager: patch usage (see feat: pre-register partial transactions in AWS Lambda #3285 (comment))
  • See fix: support Lambda instrumentation for contextManager: 'patch'; refactor Lambda tests #3305 for this.
• Consider a separate issue to look at replacing lambda-local usage for local testing with https://github.com/aws/aws-lambda-runtime-interface-emulator that Jonas has been looking at for the Java agent. It runs the AWS-provided emulator in a Docker container, allows using the real Lambda extension. It doesn't support the Logs API and not sure about the newer Telemetry API, so it isn't a perfect representation, but is a lot closer. Java agent PR: Added AWS Lambda integration test apm-agent-java#3120
• Add a follow-up issue (to lambda wrapper doesn't handle uncaughtException or unhandleRejection #2379) to possibly do better with reporting an error APM object for unhandleException. With this change we'll be reporting a transaction with transaction.outcome = "failure", which is good, but there won't be an "error" object with a stacktrace that points to the location of the throw error.
  • See lambda wrapper doesn't handle uncaughtException or unhandleRejection #2379 (comment). I didn't open a new issue.

[trentm]

A note on support for contextManager: patch usage

To support the "synchronously pre-register a transaction with the Lambda extension" behaviour change, were are necessarily using Promises and await ... in the wrappedLambdaHandler instrumentation now. This breaks async context tracking for the contextManager: patch config option usage. The "patch" context manager is deprecated, so this isn't considered a big deal. However, it can be supported again via this diff:

diff --git a/lib/lambda.js b/lib/lambda.js
index 870c0b4f..df27d71a 100644
--- a/lib/lambda.js
+++ b/lib/lambda.js
@@ -438,6 +438,7 @@ function setS3SingleData (trans, event, context, faasId, isColdStart) {

function elasticApmAwsLambda (agent) {
  const log = agent.logger
+  const ins = agent._instrumentation

  /**
    * Register this transaction with the Lambda extension, if possible.  This
@@ -644,13 +645,14 @@ function elasticApmAwsLambda (agent) {
      // Note: Wrapping context needs to happen *before any `await` calls* in
      // this function, otherwise the Lambda Node.js Runtime will call the
      // *unwrapped* `context.{succeed,fail,done}()` methods.
+      const runContext = ins.currRunContext()
      wrapContext(trans, event, context, triggerType)
      const wrappedCallback = wrapLambdaCallback(trans, event, context, triggerType, callback)

      await registerTransaction(trans, context.awsRequestId)

      try {
-        var retval = fn.call(this, event, context, wrappedCallback)
+        const retval = ins.withRunContext(runContext, fn, this, event, context, wrappedCallback)
        if (retval instanceof Promise) {
          return retval
        } else {

or similar.

The issue with applying that diff right now is that it adds usage of agent._instrumentation in this instrumentation, which is good and fine, but breaks the AgentMock being used in a large subset of test/lambda/*.test.js. It'll involve either another fake bandaid to AgentMock to have it start simulating a subset of agent._instrumentation.* behaviour or hundreds of lines of changes to the lambda tests to remove usage of AgentMock. I want to do the latter.

However, I want to do the latter in a separate change to not add noise to this change.

[trentm]

However, I want to do the latter in a separate change to not add noise to this change.

#3305 for this