New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to installation instructions on Linux #256

Closed
TimoRiikonen opened this Issue Mar 20, 2016 · 3 comments

Comments

Projects
None yet
2 participants
@TimoRiikonen

TimoRiikonen commented Mar 20, 2016

You can find Linux installation instructions fixes on below.

Install G++

With Fedora and Red Hat Linux the installation command is

    sudo dnf install gcc

If you have it installed already, then you will get acknowledgment “Nothing to do”.

Install ACML

Download this file: acml-5-3-1-ifort-64bit.tgz
Install as instructed in CNTK installation instructions, except replace "make install" with command

    sudo make install

You can combine various .bashrc changes to together as:

    export ACML_FMA=0
    export LD_LIBRARY_PATH=/usr/local/mpi/lib:/opt/acml5.3.1/ifort64/lib:/opt/acml5.3.1/ifort64_mp/lib:$LD_LIBRARY_PATH
    export PATH=/usr/local/mpi/bin:~/Repos/CNTK/build/production/bin/:$PATH

Installation

Follow Setup-CNTK-on-Linux except the directory name is in uppercase as it is defined in GIT:

    cd ~/Repos/CNTK

If you have 8 GB memory or less then you must use regular compilation command “make all” instead of “make -j all”. The latter is tens of times slower as the memory amount is not sufficient for concurrent compilation. Concurrent compilation also gave me an error which I didn't get when compiling with a single process.

    cd ~/Repos/CNTK/Examples/Other/Simple2d/Data
    cntk configFile=../Config/Simple.cntk

Run the test cases as instructed. CPU test case gives one more line than documented.

@alexeyo26

This comment has been minimized.

Show comment
Hide comment
@alexeyo26

alexeyo26 Mar 20, 2016

Member

Timo, thank you for these updates. A couple of comments.

ACML. I fixed the error regarding sudo make install

Installation directory. I fixed the text in a way that the CNTK folder will be always in small letters, i.e. cntk

On make -j. I updated setup instructions just a couple of days ago and specifically put a section on make -j Would you care to look at it and tell me whether it's clear enough. (I did not put "a cut off" of 8GB memory, because we saw make -j working even with a smaller RAM and failing with bigger)

Thank you,
Alexey

P.S. As you saw we describe the specific commands for Ubuntu. The reason is simple - it is the system we build and test on. I plan to review Linux setup instructions one day and update ALL sections with other popular distribution specific commands. So thank you once again.

Member

alexeyo26 commented Mar 20, 2016

Timo, thank you for these updates. A couple of comments.

ACML. I fixed the error regarding sudo make install

Installation directory. I fixed the text in a way that the CNTK folder will be always in small letters, i.e. cntk

On make -j. I updated setup instructions just a couple of days ago and specifically put a section on make -j Would you care to look at it and tell me whether it's clear enough. (I did not put "a cut off" of 8GB memory, because we saw make -j working even with a smaller RAM and failing with bigger)

Thank you,
Alexey

P.S. As you saw we describe the specific commands for Ubuntu. The reason is simple - it is the system we build and test on. I plan to review Linux setup instructions one day and update ALL sections with other popular distribution specific commands. So thank you once again.

@TimoRiikonen

This comment has been minimized.

Show comment
Hide comment
@TimoRiikonen

TimoRiikonen Mar 20, 2016

Ah, I didn't know you can change the casing so easily with git, it works correct now.

Yes, I noticed your commands were for Ubuntu, but it is of course nice to mention if the framework is likely to work even when you don't officially support them. Fedora at least compiled and ran the basic version.

Also I would recommend to change the recommended compilation on first attempt to "make -j 3" to avoid situations like I got. My settings are "Intel® Core™ i7-4500U CPU @ 1.80GHz × 4" with 8 GB memory and without SDD drive which should be very typical for a testing and development environment. That means it has four threads, so the user is still able to kill the process if the compilation takes hours instead of minutes. This environment has too much processor capacity vs. memory + disk, which creates an outcome where each thread reads the hard drive replacing from cache something critical from other threads. So running concurrently was dozens of times slower than running on one thread only.

TimoRiikonen commented Mar 20, 2016

Ah, I didn't know you can change the casing so easily with git, it works correct now.

Yes, I noticed your commands were for Ubuntu, but it is of course nice to mention if the framework is likely to work even when you don't officially support them. Fedora at least compiled and ran the basic version.

Also I would recommend to change the recommended compilation on first attempt to "make -j 3" to avoid situations like I got. My settings are "Intel® Core™ i7-4500U CPU @ 1.80GHz × 4" with 8 GB memory and without SDD drive which should be very typical for a testing and development environment. That means it has four threads, so the user is still able to kill the process if the compilation takes hours instead of minutes. This environment has too much processor capacity vs. memory + disk, which creates an outcome where each thread reads the hard drive replacing from cache something critical from other threads. So running concurrently was dozens of times slower than running on one thread only.

@alexeyo26

This comment has been minimized.

Show comment
Hide comment
@alexeyo26

alexeyo26 Mar 20, 2016

Member

Timo, thank you indeed, but we are being even more conservative and propose using make -j 2 which is working even on virtual machines.

Again, thank you.

Member

alexeyo26 commented Mar 20, 2016

Timo, thank you indeed, but we are being even more conservative and propose using make -j 2 which is working even on virtual machines.

Again, thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment