Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate memory leaks in code and tests #542

Closed
tegefaulkes opened this issue Aug 3, 2023 · 1 comment
Closed

Investigate memory leaks in code and tests #542

tegefaulkes opened this issue Aug 3, 2023 · 1 comment
Assignees
Labels
development Standard development r&d:polykey:supporting activity Supporting core activity

Comments

@tegefaulkes
Copy link
Contributor

tegefaulkes commented Aug 3, 2023

Specification

During #534 we discovered some potential memory leaks. The Ci job for the tests/client domain was running out of memory and core dumping. There seems to be a problem here.

Running the client domain tests with npm run test -- --runInBand --logHeapUsage tests/client shows a steady increase of heap usage between tests totalling 1.2GB of heap used by the end.

A stopgap measure has been taken by running jest with the node option --expose-cg allowing jest to trigger garbage collection between tests. 52 test files have been combined into 7 files, one for each domain. This is enough to solve the problem for now.

Given that reducing the number of files significantly reduced the memory usage, this suggests that the main memory leak is related to the global scope and imports in some fashion.

Further work needs to be done to discover these leaks and resolve them.

Additional context

Tasks

  1. Investigate the source of the leaks.
  2. Do A|B testing per domain to determine where the majority of the memory is being used
@tegefaulkes tegefaulkes added the development Standard development label Aug 3, 2023
@tegefaulkes tegefaulkes self-assigned this Aug 3, 2023
@tegefaulkes tegefaulkes mentioned this issue Aug 3, 2023
13 tasks
@CMCDragonkai
Copy link
Member

I'm going to close this for now, because this issue needs to be a bit more specific about what exactly is running out of memory. And #551 will be the one looking into this. I'll just roll this into the general testing required there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development Standard development r&d:polykey:supporting activity Supporting core activity
Development

No branches or pull requests

2 participants