Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Memory Leak] on module loading #6738

Closed
siluri opened this issue Jul 23, 2018 · 9 comments
Closed

[Memory Leak] on module loading #6738

siluri opened this issue Jul 23, 2018 · 9 comments
Labels

Comments

@siluri
Copy link

siluri commented Jul 23, 2018

🐛 Bug Report

Jest no longer releases memory for specific modules - e.g. fs-extra | graceful-fs | archiver

To Reproduce

make a testcase which loads one of the moduls and repeat that. so u can see the heapusage increase or u use my repo for that

Expected behavior

memory decrease after testcase

Link to repo

Testcases to reproduce Memory Leak easily include documented test results.

System:
    OS: Windows 10
    CPU: x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
  Binaries:
    Yarn: 1.7.0 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
    npm: 5.6.0 - C:\Program Files\nodejs\npm.CMD

Test Result of my repo

✔️result without one of "leaky" moduls.

$ node --expose-gc ./node_modules/jest/bin/jest --runInBand --no-cache --logHeapUsage
 PASS  __tests__/leak.test.js (1).js (40 MB heap size)
 PASS  __tests__/leak.test.js (11).js (40 MB heap size)
 PASS  __tests__/leak.test.js (2).js (40 MB heap size)
 PASS  __tests__/leak.test.js (4).js (40 MB heap size)
 PASS  __tests__/leak.test.js (10).js (40 MB heap size)
 PASS  __tests__/leak.test.js (5).js (40 MB heap size)
 PASS  __tests__/leak.test.js (6).js (40 MB heap size)
 PASS  __tests__/leak.test.js (8).js (40 MB heap size)
 PASS  __tests__/leak.test.js (7).js (40 MB heap size)
 PASS  __tests__/leak.test.js (9).js (40 MB heap size)
 PASS  __tests__/leak.test.js (3).js (40 MB heap size)

Test Suites: 11 passed, 11 total
Tests:       11 passed, 11 total
Snapshots:   0 total
Time:        4.134s
Ran all test suites.
Done in 5.05s.

❌result with fs-extra

$ node --expose-gc ./node_modules/jest/bin/jest --runInBand --no-cache --logHeapUsage
 PASS  __tests__/leak.test.js (1).js (40 MB heap size)
 PASS  __tests__/leak.test.js (11).js (43 MB heap size)
 PASS  __tests__/leak.test.js (2).js (46 MB heap size)
 PASS  __tests__/leak.test.js (3).js (50 MB heap size)
 PASS  __tests__/leak.test.js (10).js (53 MB heap size)
 PASS  __tests__/leak.test.js (5).js (56 MB heap size)
 PASS  __tests__/leak.test.js (6).js (59 MB heap size)
 PASS  __tests__/leak.test.js (7).js (62 MB heap size)
 PASS  __tests__/leak.test.js (9).js (65 MB heap size)
 PASS  __tests__/leak.test.js (8).js (68 MB heap size)
 PASS  __tests__/leak.test.js (4).js (71 MB heap size)

Test Suites: 11 passed, 11 total
Tests:       11 passed, 11 total
Snapshots:   0 total
Time:        4.788s
Ran all test suites.
Done in 5.71s.

❌result with graceful-fs

$ node --expose-gc ./node_modules/jest/bin/jest --runInBand --no-cache --logHeapUsage
 PASS  __tests__/leak.test.js (1).js (40 MB heap size)
 PASS  __tests__/leak.test.js (11).js (43 MB heap size)
 PASS  __tests__/leak.test.js (2).js (46 MB heap size)
 PASS  __tests__/leak.test.js (3).js (49 MB heap size)
 PASS  __tests__/leak.test.js (10).js (52 MB heap size)
 PASS  __tests__/leak.test.js (5).js (55 MB heap size)
 PASS  __tests__/leak.test.js (6).js (58 MB heap size)
 PASS  __tests__/leak.test.js (7).js (61 MB heap size)
 PASS  __tests__/leak.test.js (8).js (64 MB heap size)
 PASS  __tests__/leak.test.js (9).js (66 MB heap size)
 PASS  __tests__/leak.test.js (4).js (69 MB heap size)

Test Suites: 11 passed, 11 total
Tests:       11 passed, 11 total
Snapshots:   0 total
Time:        4.682s
Ran all test suites.
Done in 5.57s.

❌result with node fs & archiver

$ node --expose-gc ./node_modules/jest/bin/jest --runInBand --no-cache --logHeapUsage
 PASS  __tests__/leak.test.js (1).js (43 MB heap size)
 PASS  __tests__/leak.test.js (2).js (47 MB heap size)
 PASS  __tests__/leak.test.js (3).js (51 MB heap size)
 PASS  __tests__/leak.test.js (4).js (54 MB heap size)
 PASS  __tests__/leak.test.js (11).js (58 MB heap size)
 PASS  __tests__/leak.test.js (6).js (62 MB heap size)
 PASS  __tests__/leak.test.js (7).js (65 MB heap size)
 PASS  __tests__/leak.test.js (8).js (69 MB heap size)
 PASS  __tests__/leak.test.js (9).js (72 MB heap size)
 PASS  __tests__/leak.test.js (10).js (76 MB heap size)
 PASS  __tests__/leak.test.js (5).js (80 MB heap size)

Test Suites: 11 passed, 11 total
Tests:       11 passed, 11 total
Snapshots:   0 total
Time:        5.335s
Ran all test suites.
Done in 6.26s.
@siluri
Copy link
Author

siluri commented Jul 23, 2018

test with module fast-glob

✔️result with node fs & fast-glob

$ node --expose-gc ./node_modules/jest/bin/jest --runInBand --no-cache --logHeapUsage
 PASS  __tests__/leak.test.js (1).js (43 MB heap size)
 PASS  __tests__/leak.test.js (11).js (43 MB heap size)
 PASS  __tests__/leak.test.js (2).js (43 MB heap size)
 PASS  __tests__/leak.test.js (3).js (43 MB heap size)
 PASS  __tests__/leak.test.js (10).js (46 MB heap size)
 PASS  __tests__/leak.test.js (5).js (43 MB heap size)
 PASS  __tests__/leak.test.js (7).js (43 MB heap size)
 PASS  __tests__/leak.test.js (6).js (43 MB heap size)
 PASS  __tests__/leak.test.js (9).js (43 MB heap size)
 PASS  __tests__/leak.test.js (8).js (43 MB heap size)
 PASS  __tests__/leak.test.js (4).js (46 MB heap size)

Test Suites: 11 passed, 11 total
Tests:       11 passed, 11 total
Snapshots:   0 total
Time:        5.47s
Ran all test suites.
Done in 6.42s.

@kirillgroshkov
Copy link

Wow, sounds bad! Any update/progress on that? We noticed that in our project where we have ~50 test files, it started to fail in CircleCI because of memory usage. We use fs-extra in our code.

@siluri
Copy link
Author

siluri commented Oct 10, 2018

@kirillgroshkov workaround for your issue is: change fs-extra to fs-jetpack

@kirillgroshkov
Copy link

Well, I'm not that easy on just changing dependencies that we use in many projects extensively (fs-extra) to dependency I've never seen before. I'll check it out, of course, thanks! But more likely I'll change to native Node's fs in some problematic cases.

Interesting that we use fs-extra in production and we don't have a memory leak for days and days. It only appears in Jest.

@siluri
Copy link
Author

siluri commented Oct 10, 2018

Yes, fs-extra is not leaky. It declare global stuff and every test in jest will declare it again and that leaks. Fs-jetpack don't do that. If u find a solution to cleanup up the global stuff, let me know.

@siluri
Copy link
Author

siluri commented Oct 10, 2018

@kirillgroshkov here u see the result of node fs native. But we need fs async - so we use fs-jetpack.

Checkout #6814

@github-actions
Copy link

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 14 days.

@SimenB
Copy link
Member

SimenB commented Mar 3, 2022

From what I can tell, this is an issue with those modules, not Jest. Or, Jest probably exposes/excarebates/creates the condition for them to leak, but still. graceful-fs was fixed a couple of years ago, the others still have the issue. But I'd say this is a duplicate of #6814 (which has some details about why this happens).

@SimenB SimenB closed this as completed Mar 3, 2022
@github-actions
Copy link

github-actions bot commented Apr 3, 2022

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 3, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

3 participants