Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vcpkg] Optionally limit the number of cores during vcpkg bootstrap #19010

Merged
merged 2 commits into from Jul 27, 2021

Conversation

jherico
Copy link
Contributor

@jherico jherico commented Jul 19, 2021

Add an option to the bash bootstrap script that allows a user to force the use of only a single CPU during both compilation and linking of the vcpkg executable.

Building on low-memory / high-core count systems like 1-2 GB variants of Raspberry PIs will often fail because of memory exhaustion. Each compile will typically take a few hundred MB and running 4 of them concurrently will easily exhaust even a 2GB setup. Further, such systems often have no swap space configured. This command line option ensures that vcpkg will bootstrap successfully, albeit slowly, even when only 1GB of ram is available.

This PR does not attempt to address any similar issues that might arise from building packages using vcpkg

  • What does your PR fix?

Fixes #14534

  • Which triplets are supported/not supported? Have you updated the [CI baseline]

This is a modification to the bootstrap script and should have no impact on the triplets, only a potential impact on the speed of bootstrapping.

Yes. The vast majority of the guide doesn't apply since this is a change to the boostrapping mechanism, not to any port or vcpkg functionality.

  • If you have added/updated a port: Have you run ./vcpkg x-add-version --all and committed the result?

N/A

@JackBoosY JackBoosY requested a review from vicroms July 20, 2021 01:01
@JackBoosY JackBoosY added category:vcpkg-feature The issue is a new capability of the tool that doesn’t already exist and we haven’t committed requires:vcpkg-team-review This PR or issue requires someone on the vcpkg team to take a further look. labels Jul 20, 2021
@aizuon
Copy link
Contributor

aizuon commented Jul 20, 2021

Could this feature also be implemented when compiling packages? Would be nice to leave half of the cores unattended when compiling big packages, i.e. Qt, to do other work to spend time.

@PhoebeHui PhoebeHui changed the title Optionally limit the number of cores during vcpkg bootstrap [vcpkg] Optionally limit the number of cores during vcpkg bootstrap Jul 20, 2021
@jherico
Copy link
Contributor Author

jherico commented Jul 20, 2021

@aizuon It would be inconsistent. As far as I understand, not every package uses CMake for the build wrapper, and the approach here only works with CMake. It's likely that there may be some necessary similar feature that would be useful for package building, but this PR is just focused on the bootstrap process and unblocking people in the situation described above.

@vicroms
Copy link
Member

vicroms commented Jul 23, 2021

@aizuon You can try setting the VCPKG_MAX_CONCURRENCY environment variable which most buildsystem helper functions support (there may be some ports with weird buildsystems that won't).

@jherico This LGTM. The team may have some opinions about the argument name, I'll bring it up in today's meeting.

@ras0219-msft
Copy link
Contributor

I feel like it would make more sense to just make bootstrap respect VCPKG_MAX_CONCURRENCY

@jherico
Copy link
Contributor Author

jherico commented Jul 25, 2021

@ras0219 I've modified the build script to respect VCPKG_MAX_CONCURRENCY and removed the other modifications. I feel like it would be helpful to have some text in the -help output but I couldn't come up with something both informative and concise, so I left it as is.

@JackBoosY JackBoosY added the info:reviewed Pull Request changes follow basic guidelines label Jul 26, 2021
@vicroms vicroms removed the requires:vcpkg-team-review This PR or issue requires someone on the vcpkg team to take a further look. label Jul 26, 2021
@vicroms vicroms merged commit 593c26c into microsoft:master Jul 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:vcpkg-feature The issue is a new capability of the tool that doesn’t already exist and we haven’t committed info:reviewed Pull Request changes follow basic guidelines
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Building vcpkg on RPi fails with out of memory. Add number of cores argument option to bootstrap script
5 participants