Async stack traces for callback errors

[aalexgabi]

What problem are you trying to solve?

I need to know the caller that generated a given http error.

Describe the feature

I use node 13 which has async stack traces but I don't have the caller stack trace when using got. Instead I get this stack:

    err HTTPError: Response code 404 (Not Found)
        at EventEmitter.<anonymous> (.../node_modules/got/dist/
source/as-promise.js:118:31)
        at processTicksAndRejections (internal/process/task_queues.js:97:5) {
      name: 'HTTPError'
    }

I guess this happens because the connection can be used by multiple callers but there should be a way to have the whole stack.

[szmarczak]

This works as expected: EventEmitter.<anonymous> is the handler for the response event received from the native Node.js call.

[szmarczak]

Related: #795

[szmarczak]

I don't think we can do much about this. This is just how Node.js works. One possible solution would be to create a stacktrace before making the request and create another one when erroring and then merge these two.

@sindresorhus What do you think?

[sindresorhus]

Could we use async_hooks somehow to improve the stack traces?

• https://stackoverflow.com/questions/45359445/are-there-any-npm-modules-that-use-async-hooks-to-retrieve-async-stack-trace
• https://stackoverflow.com/questions/54914770/is-there-a-good-way-to-surface-error-traces-in-production-across-event-emitters

[szmarczak]

We could, but:

1. the API is marked experimental so it will give a warning (it's live since Node.js 8.1) - but I think it's stable for this use case as I don't see any issues that would break this
2. the change would be visible globally.

I think we should create another .md file and show how to achieve the desired result step-by-step.

[aalexgabi]

One possible solution would be to create a stacktrace before making the request and create another one when erroring and then merge these two.

@szmarczak This approach works. Our team used in many projects that involved multiplexing a connection or a event emitter. The only think I would worry about is the CPU load associated with generating the stack.

Maybe stack trace generation performance has improved in the last years. I have seen some articles talking about zero cost async stack traces but I'm not sure if it applies:

• https://v8.dev/docs/stack-trace-api
• https://docs.google.com/document/d/13Sy_kBIJGP0XT34V1CV3nkWya4TwYx9L3Yv45LdGB6Q/edit

[szmarczak]

@aalexgabi The zero-cost stacktraces you mentioned are related to async/await (and are already on by default), not I/O operations.

[sindresorhus]

the change would be visible globally.

Yeah, we cannot modify anything globally.

But would it have to be global? I was thinking of storing the stack in the async context and then merge it on error.

I think we should create another .md file and show how to achieve the desired result step-by-step.

👍

[szmarczak]

But would it have to be global? I was thinking of storing the stack in the async context and then merge it on error.

That would work too, but the question is how much performance you would have to give up, since on every async task there would be called Error.captureStackTrace...

[sindresorhus]

Maybe we should just make an independent module that creates long stack traces using async hooks contexts and then recommend that, but warn that it might have a performance impact. Then the user can decide whether it's worth it or not.

[szmarczak]

async_hooks is now [almost] stable >=13.10.0 https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V13.md/#notable-changes-1

[PSeitz]

Would the --async-stack-traces option in nodejs12 solve this issue?
http://thecodebarbarian.com/async-stack-traces-in-node-js-12.html

[szmarczak]

@PSeitz It's already on by default IIRC.

[rightaway]

@szmarczak Is there any workaround for this even if it makes performance worse? Trying to debug some issues and the stack traces of where the request was initially made from would really help.

[szmarczak]

You can capture the stack trace in a handler and store it inside options.context.stack.

[szmarczak]

import got from 'got';

const instance = got.extend({
	handlers: [
		(options, next) => {
			Error.captureStackTrace(options.context);

			return next(options);
		}
	],
	hooks: {
		beforeError: [
			error => {
				error.source = error.options.context.stack.split('\n');

				return error;
			}
		]
	}
});

try {
	await instance('https://httpbin.org/status/400');
} catch (error) {
	console.error(error);
}

[szmarczak]

https://github.com/sindresorhus/got/blob/main/documentation/async-stack-traces.md#conclusion

[jgehrcke]

@szmarczak thank you for crafting this document!

[bencmbrook]

@szmarczak I'm using that snippet but getting "Cannot add property stack, object is not extensible" when setting stack on options.context via Error.captureStackTrace(options.context);. Does this have an Object.freeze / .preventExtensions / .seal now?

[szmarczak]

You're right, indeed. https://github.com/sindresorhus/got/blame/main/source/core/options.ts#L2475

This should be removed when #1459 was merged. I just ran the example from the docs and for some reason it worked, but that's weird. Can you send me a repro?

[bencmbrook]

I can't send a full repro at the moment, but I think it's likely you're not in strict mode ('use strict';) if you're not getting the TypeError

FWIW, I worked around this with

const stackTraceHandler = (options, next) => {
  const context = {};
  Error.captureStackTrace(context, stackTraceHandler);
  options.context = { ...options.context, stack: context.stack };
  return next(options);
};

...

got.extend({
  handlers: [stackTraceHandler],
  // ...
});

[bencmbrook]

Also in case it's helpful to any future readers using TypeScript, here's my working TypeScript version

import got, { BeforeErrorHook, HandlerFunction, RequestError } from 'got';

const stackTraceHandler: HandlerFunction = (options, next) => {
  const context: { stack?: string } = {};
  Error.captureStackTrace(context, stackTraceHandler);
  options.context = { ...options.context, stack: context.stack };
  return next(options);
};

const addSourceStackTraceToError: BeforeErrorHook = (error: RequestError) => {
  error.stack = `${error.stack}\n---Source Stack---\n${error.options.context.stack}`;
  return error;
};

export default got.extend({
  handlers: [stackTraceHandler],
  hooks: {
    beforeError: [addSourceStackTraceToError],
  },
});

Adding the source property to the error object was causing issues so I just decided to append to the existing stack.

[hvrauhal]

It would be great if the above typescript version was included in the document https://github.com/sindresorhus/got/blob/main/documentation/async-stack-traces.md#conclusion