Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-core assembly in debian stretch #6

Closed
isopen opened this issue Jan 7, 2018 · 7 comments
Closed

Multi-core assembly in debian stretch #6

isopen opened this issue Jan 7, 2018 · 7 comments

Comments

@isopen
Copy link
Contributor

isopen commented Jan 7, 2018

You should not experiment with fewer cores.

Debian stretch. I have 8 cores in my car.

I did the following:
zlib from the source code https://packages.debian.org/source/stretch/zlib
apt install libssl-dev libreadline-dev ccache
other packages according to the instructions

first step:
mkdir build
cd build
cmake ..
cmake --build . (at this step I was dejected)
ctrl+c

second step
make -j 6
the virtual machine hangs. restarting the virtual machine.

third step
make -j 4
cautiously somewhere in the middle the system ceases to hang and everything is going to.
just in case cmake --build .

Do not forget about the ccache package for subsequent assemblies useful.
Then you can cmake --build .

Are we going to collect in several cores or will we wait a long time in one core according to the instructions?

@arseny30
Copy link
Contributor

arseny30 commented Jan 7, 2018

It probably hangs because all RAM is used. You may try clang instead of g++, it uses much less memory than g++ and compiles TDLib quicker.

Also cmake -GNinja .. will use ninja instead of make. ninja will try to determine needed number concurrent jobs by itself, so simple cmake --build . will be enough to run a build as fast as possible.

@isopen
Copy link
Contributor Author

isopen commented Jan 7, 2018

Ok :)

apt install ninja-build
cmake -GNinja ..
cmake --build .
image
Accordingly, the virtual machine hung.
Something went wrong...+ eats 10gb of ozu.
The effect is similar to make -j 6

apt install clang
CXX=clang++ CC=clang cmake ..
cmake --build . (the effect is the same as g ++)

make -j 4 although it hangs in the middle of the assembly, but collects in all cases faster :) for simple reasons.
image

@arseny30
Copy link
Contributor

arseny30 commented Jan 8, 2018

Ok, ninja won't help here. But it still looks that you just have not enough RAM for concurrent build. Am I right?

@isopen
Copy link
Contributor Author

isopen commented Jan 8, 2018

I do not have much RAM, but there is 16gb ozu.)
Library at assembly is eating nenuchutku. And all resources in general.
I may be in something I do not understand, but it's a kapets.
In 4 cores going 20+ minutes. eats whole RAM.

The moment that in our market most have only 8GB of RAM and 4 cores :))

All CPU time fills the ninja in the first place.

64GB RAM is what you need?)

@levlam
Copy link
Contributor

levlam commented Jan 8, 2018

4 GB of RAM should be enough for GCC (at least for Release builds). Clang usually needs only 1,5-2 GB of RAM for each file. You need to adjust job count to not use more RAM than available in the system. It is strange that modern C++ compilers are so slow and need so much RAM to compile simple source files, but it is not a library issue.

@isopen isopen closed this as completed Jan 8, 2018
@isopen
Copy link
Contributor Author

isopen commented Jan 8, 2018

@levlam thank

@isopen isopen reopened this Jan 8, 2018
@isopen
Copy link
Contributor Author

isopen commented Jan 8, 2018

Nevertheless, if you do not mind, we will continue.

Assembly on 4 cores of 8 and almost 9GB of RAM

  • Further all ok (1,5-3 GB)

  • At this point, the assembly eats about 5GB

image

  • Further all ok (1,5-3 GB)

  • At this place begins to eat the whole RAM

image

  • Accordingly, the operating system hangs for a while.

image
Building CXX object CMakeFiles/tdcore.dir/td/telegram/Payments.cpp.o

  • Further all ok (1,5-3 GB)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants