Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit the number of threads (fix memory usage) #5617

Closed
wants to merge 1 commit into from

Conversation

aminya
Copy link
Contributor

@aminya aminya commented Jan 9, 2021

↪️ Pull Request

See this comment for the analysis behind this PR.

Fixes #5072
Fixes #4628

💻 Examples

Parcel does not utilize all the threads it spawns so there is no point in using long-lived workers when they do not do anything. This results in huge memory usage and slow-down.

For example, here as you see, other than workers 0 and 1, the others almost do nothing.
image

🚨 Test instructions

✔️ PR Todo

  • Added/updated unit tests for this change
  • Filled out test instructions (In case there aren't any unit tests)
  • Included links to related issues/PRs

@height
Copy link

height bot commented Jan 9, 2021

Link Height tasks by mentioning a task ID in the pull request title or description, commit messages, or comments.

💡Tip: You can also use "Close T-X" to automatically close a task when the pull request is merged.

@aminya aminya mentioned this pull request Jan 9, 2021
@aminya aminya changed the title Limit the number of threads that are spawned Limit the number of threads + Fix detectRealCores for windows Jan 9, 2021
@aminya
Copy link
Contributor Author

aminya commented Jan 9, 2021

@mischnic @devongovett As I mentioned in this comment, using workers has no measurable benefits. However, if we want to keep that discussion open, I can move the limiting part to another PR, and in this PR I just fix the correct core number detection for Windows. This core number detection should fix the linked issues on windows.

@mischnic
Copy link
Member

I can move the limiting part to another PR, and in this PR I just fix the correct core number detection for Window

Please do that

@aminya aminya changed the title Limit the number of threads + Fix detectRealCores for windows Limit the number of threads Jan 13, 2021
@aminya aminya mentioned this pull request Jan 13, 2021
3 tasks
@aminya aminya force-pushed the limit-cpus branch 2 times, most recently from c8092e9 to 3810b02 Compare January 17, 2021 07:15
@aminya aminya changed the title Limit the number of threads Limit the number of threads (fix memory usage) Jan 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants