Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(node): Convert debugging code to callbacks to fix memory leak in LocalVariables integration #7637

Merged
merged 4 commits into from Mar 29, 2023

Conversation

timfish
Copy link
Collaborator

@timfish timfish commented Mar 28, 2023

Closes #7230

This PR converts the LocalVariables debugger code to callbacks so that we can call Debugger.resume synchronously from the pause callback which fixes the memory leak.

It's worth noting that now there is no usage of async, it will not be possible to use async import to improve this:

TODO: We really should get rid of this require statement below for a couple of reasons:
1. It makes the integration unusable in the SvelteKit SDK, as it's not possible to use `require`
in SvelteKit server code (at least not by default).
2. Throwing in a constructor is bad practice

I created a basic Express server to test these changes and load tested an endpoint with a caught/handled error to see if any memory is leaked. Without the async code, performance is also much improved:

Before After
Requests/sec 40 220
Av. Latency 90ms 40ms
Peak Memory 570 MB 130 MB

@timfish timfish changed the title fixUse callbacks rather than async fix(node): Convert debugging code to callbacks to fix memory leak in LocalVariables integration Mar 28, 2023
Copy link
Member

@mydea mydea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! We can think about using await import later, for now it's more important to fix/unblock this IMHO 👍

popped(result);
} catch (_) {
// If there is an error, we still want to call the complete callback
complete(result);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we clear the callbacks list here? Otherwise there is a risk of complete being called multiple times right?

Copy link
Collaborator Author

@timfish timfish Mar 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it can only get called twice if a function passed to add calls next before throwing and in every current usage next is called last.

Maybe add a throw if complete is called and there are still callbacks remaining?

}

function next(result: T): void {
const popped = callbacks.pop() || complete;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we no-op if callbacks is empty? Perhaps give some kind of indicator of success (boolean return) from the next function?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we add complete to the top of the callbacks stack, the only way next can be called when callbacks is empty is if you call next more than once per added closure or manage to call next in complete (which would be tricky to do since you have to supply complete before next is returned). The || callback here is mainly to keep the types happy and does not get hit currently.

If we don't expect to hit this path, maybe it would be better to throw if pop returns undefined?

packages/node/src/integrations/localvariables.ts Outdated Show resolved Hide resolved
@AbhiPrasad AbhiPrasad merged commit 8a25d00 into getsentry:develop Mar 29, 2023
41 checks passed
@timfish timfish deleted the fix/local-variables branch March 29, 2023 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memory Leak in LocalVariables integration
3 participants