Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the needed memory for compilation #1516

Closed
wants to merge 1 commit into from

Conversation

ellert
Copy link
Contributor

@ellert ellert commented Jan 16, 2018

The linking of rootcling_stage1 and libCling requires a lot of memory.
Since these are linked from mostly the same objects, the build is ready
to link them at the same time. If you make a parallel build this means
that the two targets that require the most amount of memory are being
linked in parallel. This exhausts the available memory, and the
computer starts swapping.

This adds a dependency of one of the targets to the other. The dependency is
not really there since it is not needed for building, but it prevents the
two memory consuming targets to be built in parallel.

A similar dependency existed before the code latest code changes
(see commit 2638f6f), then between
rootcling and libCling.

The linking of rootcling_stage1 and libCling requires a lot of memory.
Since these are linked from mostly the same objects, the build is ready
to link them at the same time. If you make a parallel build this means
that the two targets that require the most amount of memory are being
linked in parallel. This exhausts the available memory, and the
computer starts swapping.

This adds a dependency of one of the targets to the other. The dependency is
not really there since it is not needed for building, but it prevents the
two memory consuming targets to be built in parallel.

A similar dependency existed before the code latest code changes
(see commit 2638f6f), then between
rootcling and libCling.
@phsft-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@amadio
Copy link
Member

amadio commented Jan 17, 2018

@phsft-bot build

@phsft-bot
Copy link
Collaborator

Starting build on centos7/gcc49, mac1012/native, slc6/gcc49, slc6/gcc62, slc6/gcc62, ubuntu14/native, ubuntu14/native, windows10/vc15 with flags -Dvc=OFF -Dimt=ON -Dccache=ON
How to customize builds

@amadio
Copy link
Member

amadio commented Jan 22, 2018

I am sorry, but I do not think we can merge this. The artificial dependency also means that things that don't need to be rebuilt if you modify files will be rebuilt after this change. My machine is not so powerful and I never had problems with this, except for when using ICC and building multiple sources of ROOT at the same time with 4 threads each.

@amadio amadio closed this Jan 22, 2018
@Axel-Naumann
Copy link
Member

@ellert does CMake not offer an order-dependency like GNU make does?

@amadio
Copy link
Member

amadio commented Jan 23, 2018

Well, on the other hand, I guess we can afford to have this one step only done in serial if needed. @Axel-Naumann, what's your opinion on this? Would you rather merge so that this stops affecting users? Now that I realized this is already there for rootcling we might as well add the same for rootcling_stage1.

@Axel-Naumann
Copy link
Member

That depends on how often we'd rebuild rootcling_stage1 even though there was no need, i.e. we rebuild only due to the artificial order dependency. Do you have an estimate, @ellert ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants