Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build problem with MPI library #198

Open
rfvander opened this issue Dec 16, 2014 · 111 comments
Open

Build problem with MPI library #198

rfvander opened this issue Dec 16, 2014 · 111 comments

Comments

@rfvander
Copy link

I downloaded Grappa and am now trying to build it, but instructions are a bit sparse. If I define symbols CC and CXX to resolve to the Intel compilers icc and icpc, respectively, I get the error message below. Obviously, my installed MPI cannot be found. I tried to fix that by setting: “export MPICC=mpiicc” but that did not work, nor did “export MPI_C=mpiicc”. There is no reference to MPI in “configure” or in “FindPackageHandleStandardArgs.cmake “. Do you have any suggestions? By the way, I also have GASNet installed, so if that is the better communication layer, I'll use that--if I can get some instructions how to do that. Thanks.

Rob

[rfvander@bar1 grappa]$ export CC=icc
[rfvander@bar1 grappa]$ export CXX=icpc
[rfvander@bar1 grappa]$ ./configure --gen=Make --mode=Release
cmake /lustre/home/rfvander/grappa -G"Unix Makefiles" -DSHMMAX=33554432 -DCMAKE_C_COMPILER=icc -DCMAKE_CXX_COMPILER=icpc -DBASE_C_COMPILER=icc -DBASE_CXX_COMPILER=icpc -DCMAKE_BUILD_TYPE=RelWithDebInfo
-- The C compiler identification is Intel 15.0.0.20140723
-- The CXX compiler identification is Intel 15.0.0.20140723
-- Check for working C compiler: /opt/intel/tools/composer_xe_2015.0.090/bin/intel64/icc
-- Check for working C compiler: /opt/intel/tools/composer_xe_2015.0.090/bin/intel64/icc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /opt/intel/tools/composer_xe_2015.0.090/bin/intel64/icpc
-- Check for working CXX compiler: /opt/intel/tools/composer_xe_2015.0.090/bin/intel64/icpc -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Boost found: 1.53.0 -- /usr
CMake Error at /usr/share/cmake/Modules/ FindPackageHandleStandardArgs.cmake:108 (message):
Could NOT find MPI_C (missing: MPI_C_LIBRARIES)
Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:315 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake/Modules/FindMPI.cmake:587 (find_package_handle_standard_args)
CMakeLists.txt:205 (find_package)

-- Configuring incomplete, errors occurred!
See also "/lustre/home/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeOutput.log".

@nelsonje
Copy link
Member

Hi Rob,

We currently support only GCC or LLVM/Clang, and OpenMPI or MVAPICH2. Intel MPI will probably work as long as it supports MPI3 non-blocking collectives, but we'll need to update our configure script to identify the correct libraries to link against (it doesn't use the mpicc/mpicxx wrappers). I'm not sure about the Intel compiler---we depend on some GCC directives which it may or may not support.

We no longer support GASNet.

If you could get us an account on a machine with Intel compiler/MPI we could probably get these things debugged.

@bholt
Copy link
Member

bholt commented Dec 16, 2014

Jacob's right, we don't use the Intel compiler, so can't make any promises.

CMake ships with some pretty weak scripts to try to find libraries like MPI. It looks like you might have better luck with the HDF FindMPI.cmake, which seems to know about Intel MPI. I think to use it, you'd have to download that FindMPI.cmake file, and add the path to it to CMAKE_MODULE_PATH (it might work to put it in environment variable, or you can pass it to CMake through our configure script:

./configure ... -- -DCMAKE_MODULE_PATH=/path/to/new/findmpi

@rfvander
Copy link
Author

Thanks, Jacob. I don’t mind building with GCC initially, but will want to use Intel compilers when in the future. That’s kind of a nice thing to do when you’re an Intel employee ☺.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, December 15, 2014 4:20 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Hi Rob,

We currently support only GCC or LLVM/Clang, and OpenMPI or MVAPICH2. Intel MPI will probably work as long as it supports MPI3 non-blocking collectives, but we'll need to update our configure script to identify the correct libraries to link against (it doesn't use the mpicc/mpicxx wrappers). I'm not sure about the Intel compiler---we depend on some GCC directives which it may or may not support.

We no longer support GASNet.

If you could get us an account on a machine with Intel compiler/MPI we could probably get these things debugged.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-67092093.

@rfvander
Copy link
Author

Thanks, Brandon. I’ll try that HDF solution. Will let you know how I fare.

Rob

From: Brandon Holt [mailto:notifications@github.com]
Sent: Monday, December 15, 2014 4:27 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Jacob's right, we don't use the Intel compiler, so can't make any promises.

CMake ships with some pretty weak scripts to try to find libraries like MPI. It looks like you might have better luck with the HDF FindMPI.cmakehttp://www.hdfgroup.org/ftp/HDF5/hdf-java/hdf-java-examples/config/cmake/FindMPI.cmake, which seems to know about Intel MPI. I think to use it, you'd have to download that FindMPI.cmake file, and add the path to it to CMAKE_MODULE_PATH (it might work to put it in environment variable, or you can pass it to CMake through our configure script:

./configure ... -- -DCMAKE_MODULE_PATH=/path/to/new/findmpi


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-67092770.

@nelsonje
Copy link
Member

Great! As I said, we'd be happy to add explicit support for the Intel compiler and MPI; we just need a way to test it.

@rfvander
Copy link
Author

There are multiple ways to do the testing with Intel tools, Jacob. The easiest would be to obtain a free trial licensehttps://software.intel.com/en-us/articles/try-buy-tools, valid for 30 days. Next best would be to sit together (virtually) to figure things out. Getting access to a machine we own is also possible, but more work. Let’s try that if the other two options do not work.

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, December 15, 2014 4:51 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Great! As I said, we'd be happy to add explicit support for the Intel compiler and MPI; we just need a way to test it.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-67095081.

@rfvander
Copy link
Author

No success with downloading HDF's FindMPI.cmake. I put it in $PWD/CMakeFiles and then defined an environment variable: "export CMAKE_MODULE_PATH=$PWD/CMakeFiles"
Running the same configure command as before gave the exact same output as before. Evidently, the standard FindMPI.cmake is used, not the new one. Explicitly setting the CMAKE_MODULE_PATH on the command line did not work either. I tried two variations, with and without prefix "D":
[rfvander@bar1 grappa]$ ./configure --gen=Make --mode=Release -DCMAKE_MODULE_PATH=$CMAKE_MODULE_PATH
./configure:35:in <main>': invalid option: -DCMAKE_MODULE_PATH=/home/rfvander/grappa/CMakeFiles (OptionParser::InvalidOption) [rfvander@bar1 grappa]$ ./configure --gen=Make --mode=Release -CMAKE_MODULE_PATH=$CMAKE_MODULE_PATH ./configure:35:in

': invalid option: -CMAKE_MODULE_PATH=/home/rfvander/grappa/CMakeFiles (OptionParser::InvalidOption)

@rfvander
Copy link
Author

I also tried with double leading hyphen --DCMAKE_MODULE_PATH and --CMAKE_MODULE_PATH, to no avail.

@bholt
Copy link
Member

bholt commented Dec 18, 2014

Sometimes with CMake you also have to make sure you delete the build directory it's operating on before trying changes like this. Sorry I can't be more helpful than that right now. I'll see if I can find some time to test it out myself later.

@rfvander
Copy link
Author

Thanks, Brandon. Yes, I delete build each time it gets created before I do a new test--most of the time configure chokes before it gets to create build, though :). Please let me know if it would help to debug this issue via desktop sharing. I did find, by looking inside configure, that there are no standard options dealing with Cmake paths.

@bholt
Copy link
Member

bholt commented Dec 18, 2014

Okay good. Yeah I mean the idea here is that this line should be passing all the extra args through to the CMake command, so that's why I think they have to go after the --.

@rfvander
Copy link
Author

Right, and that is why I also tried it with the preceding "--" but have not got that to work yet.

@rfvander
Copy link
Author

Could you take another look at the findMPI issue I have been experiencing? The only thing I need is a way to influence the priority of the paths Cmake uses for its search files. I would really like to get over this hump. You should be able to test this with any Cmake find file, doesn’t have to be MPI. I’ll be out of the office next week, but want to hit the ground running when I return. Thanks!

Rob

From: Brandon Holt [mailto:notifications@github.com]
Sent: Thursday, December 18, 2014 11:07 AM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Okay good. Yeah I mean the idea here is that this linehttps://github.com/uwsampa/grappa/blob/master/configure#L146 should be passing all the extra args through to the CMake command, so that's why I think they have to go after the --.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-67538442.

@nelsonje
Copy link
Member

The best thing to do is for us to install the intel tools here and debug---I expect as soon as we fix the MPI discovery problem we'll run into compiler flag problems. I should have time to get them installed today or Monday, and then Brandon and I can take a look.

@rfvander
Copy link
Author

Splendid, thanks, Jacob. When you get to the Intel web site, it’ll be easiest to request a trial version of the cluster software quite, which includes the compilers and MPI, and ITAC (Intel Trace Analyzer and Collector). If you need the tools for more than 30 days, let me work on this side to get you a truly free version, not just the temporary version—can’t promise I’ll succeed, though.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Friday, December 19, 2014 12:31 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

The best thing to do is for us to install the intel tools here and debug---I expect as soon as we fix the MPI discovery problem we'll run into compiler flag problems. I should have time to get them installed today or Monday, and then Brandon and I can take a look.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-67693141.

@bholt
Copy link
Member

bholt commented Dec 19, 2014

We certainly wouldn't turn away a license. If we're going to try to maintain compatibility then we'll need more than 30 days.

Surprisingly enough, I'll also be out of commission the next couple weeks. I'll see if I can find time to look into the CMake thing this afternoon, but it's something that's not Grappa-specific at all, so other forums may have the answer.

On Dec 19, 2014, at 12:35 PM, rfvander notifications@github.com wrote:

Splendid, thanks, Jacob. When you get to the Intel web site, it’ll be easiest to request a trial version of the cluster software quite, which includes the compilers and MPI, and ITAC (Intel Trace Analyzer and Collector). If you need the tools for more than 30 days, let me work on this side to get you a truly free version, not just the temporary version—can’t promise I’ll succeed, though.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Friday, December 19, 2014 12:31 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

The best thing to do is for us to install the intel tools here and debug---I expect as soon as we fix the MPI discovery problem we'll run into compiler flag problems. I should have time to get them installed today or Monday, and then Brandon and I can take a look.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-67693141.

Reply to this email directly or view it on GitHub.

@nelsonje
Copy link
Member

I've made some progress with this. I installed the eval version of (I think this is the right name) Intel Parallel Studio XE Cluster Edition 2015 Update 1 on our cluster, and managed to get Grappa code to run with Intel MPI, but not with the Intel compiler. More details:

Intel MPI with GCC 4.8

This just worked once I sourced the Intel MPI variable file, even without changing the MPI detection code in the configure script. I verified this by looking at the libraries listed in the Found MPI_C and Found MPI_CXX lines printed by the configure script. I did have to set a few environment variables to get MPI to work with our cluster's scheduler and network:

source  /sampa/share/intel-cluster-studio/impi_latest/bin64/mpivars.sh
export I_MPI_PMI_LIBRARY=/usr/lib64/libpmi.so   # to support Slurm srun
export I_MPI_FABRICS='shm:ofa'                  #  supported fabric list on our cluster

After this I was able to configure, make and run (with grappa_run) some of the demo programs. I didn't look into performance at this point.

Intel MPI with Intel compiler

I ran into both a compilation error, and once I worked around that, a segfault. The source of the segfault is not yet clear. it's showing up in a place where a segfault shouldn't be possible, but it's in the worker spawn code, and I know our stack switching code has confused other compilers in the past. I'll have to spend some more time digging into this.

I got the configure script to pick up the Intel compiler by doing this:

source /sampa/share/intel-cluster-studio/bin/iccvars.sh intel64
export CC=`which icc`
export CXX=`which icpc`

@rfvander
Copy link
Author

That is great progress, Jacob. I’ll try to duplicate your effort on my cluster. Will keep you posted.
How fun that you accomplished something impossible.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, December 29, 2014 11:12 AM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

I've made some progress with this. I installed the eval version of (I think this is the right name) Intel Parallel Studio XE Cluster Edition 2015 Update 1 on our cluster, and managed to get Grappa code to run with Intel MPI, but not with the Intel compiler. More details:

Intel MPI with GCC 4.8

This just worked once I sourced the Intel MPI variable file, even without changing the MPI detection code in the configure script. I verified this by looking at the libraries listed in the Found MPI_C and Found MPI_CXX lines printed by the configure script. I did have to set a few environment variables to get MPI to work with our cluster's scheduler and network:

source /sampa/share/intel-cluster-studio/impi_latest/bin64/mpivars.sh

export I_MPI_PMI_LIBRARY=/usr/lib64/libpmi.so # to support Slurm srun

export I_MPI_FABRICS='shm:ofa' # supported fabric list on our cluster

After this I was able to configure, make and run (with grappa_run) some of the demo programs. I didn't look into performance at this point.

Intel MPI with Intel compiler

I ran into both a compilation error, and once I worked around that, a segfault. The source of the segfault is not yet clear. it's showing up in a place where a segfault shouldn't be possible, but it's in the worker spawn code, and I know our stack switching code has confused other compilers in the past. I'll have to spend some more time digging into this.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-68289095.

@rfvander
Copy link
Author

rfvander commented Jan 1, 2015

Hi Jacob,

Unfortunately, what worked for you didn’t for me—I should have known, since I always source my Intel MPI resource files in my login shell. Changing the compilers to gcc/g++ doesn’t affect the search for the MPI library, though I did try without the Intel compilers. Still searching.
Happy New Year!

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, December 29, 2014 11:12 AM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

I've made some progress with this. I installed the eval version of (I think this is the right name) Intel Parallel Studio XE Cluster Edition 2015 Update 1 on our cluster, and managed to get Grappa code to run with Intel MPI, but not with the Intel compiler. More details:

Intel MPI with GCC 4.8

This just worked once I sourced the Intel MPI variable file, even without changing the MPI detection code in the configure script. I verified this by looking at the libraries listed in the Found MPI_C and Found MPI_CXX lines printed by the configure script. I did have to set a few environment variables to get MPI to work with our cluster's scheduler and network:

source /sampa/share/intel-cluster-studio/impi_latest/bin64/mpivars.sh

export I_MPI_PMI_LIBRARY=/usr/lib64/libpmi.so # to support Slurm srun

export I_MPI_FABRICS='shm:ofa' # supported fabric list on our cluster

After this I was able to configure, make and run (with grappa_run) some of the demo programs. I didn't look into performance at this point.

Intel MPI with Intel compiler

I ran into both a compilation error, and once I worked around that, a segfault. The source of the segfault is not yet clear. it's showing up in a place where a segfault shouldn't be possible, but it's in the worker spawn code, and I know our stack switching code has confused other compilers in the past. I'll have to spend some more time digging into this.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-68289095.

@nelsonje
Copy link
Member

nelsonje commented Jan 1, 2015

Too bad! I'll take a look at the MPI detection code and see if I can offer any hints.

If possible, could you paste in your $PATH and $LD_LIBRARY_PATH variables? Maybe we can identify an ordering problem.

Happy New Year to you too!

@rfvander
Copy link
Author

rfvander commented Jan 1, 2015

Sure, Jacob:

[rfvander@bar1 bin]$ echo $PATH
/opt/intel/tools//impi/5.0.1.035/intel64/bin:/opt/intel/tools/composer_xe_2015.0.090/bin/intel64:/opt/intel/tools/composer_xe_2015.0.090/mpirt/bin/intel64:/opt/intel/tools/composer_xe_2015.0.090/debugger/gdb/intel64_mic/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:.:/home/rfvander/.local/bin:/home/rfvander/bin
[rfvander@bar1 bin]$ echo $LD_LIBRARY_PATH
/opt/intel/tools//impi/5.0.1.035/intel64/lib:/opt/intel/tools/composer_xe_2015.0.090/compiler/lib/intel64:/opt/intel/tools/composer_xe_2015.0.090/mpirt/lib/intel64:/opt/intel/tools/composer_xe_2015.0.090/ipp/../compiler/lib/intel64:/opt/intel/tools/composer_xe_2015.0.090/ipp/lib/intel64:/opt/intel/tools/composer_xe_2015.0.090/compiler/lib/intel64:/opt/intel/tools/composer_xe_2015.0.090/mkl/lib/intel64:/opt/intel/tools/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4::/home/rfvander/xstack/ocr/install/x86-pthread-x86/lib

Please promise you won’t work on this tonight ☺.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Wednesday, December 31, 2014 4:45 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Too bad! I'll take a look at the MPI detection code and see if I can offer any hints.

If possible, could you paste in your $PATH and $LD_LIBRARY_PATH variables? Maybe we can identify an ordering problem.

Happy New Year to you too!


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-68476864.

@nelsonje
Copy link
Member

nelsonje commented Jan 1, 2015

Don't worry; going home shortly. :-)

Those variables look fine. Another thing to try: run the configure script with tracing enabled. There will be a bunch of output from the FindMPI.cmake module, and we may identify something there. Send me a copy of the output over email (just in case anything sensitive ends up in the trace). Here's the command I used to do this:

./configure -- --trace 2>&1 | tee cmake-trace.txt

@rfvander
Copy link
Author

rfvander commented Jan 5, 2015

Thanks, Jacob. I sent the output via email.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Wednesday, December 31, 2014 5:14 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Don't worry; going home shortly. :-)

Those variables look fine. Another thing to try: run the configure script with tracing enabled. There will be a bunch of output from the FindMPI.cmake module, and we may identify something there. Send me a copy of the output over email (just in case anything sensitive ends up in the trace). Here's the command I used to do this:

./configure -- --trace 2>&1 | tee cmake-trace.txt


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-68477563.

@nelsonje
Copy link
Member

nelsonje commented Jan 8, 2015

Rob, try configuring Grappa with a command like this (with the paths fixed for your Intel MPI installation, of course):

./configure -- -DMPI_C_COMPILER=/sampa/share/intel-cluster-studio/impi_latest/bin64/mpigcc -DMPI_CXX_COMPILER=/sampa/share/intel-cluster-studio/impi_latest/bin64/mpigxx

@nelsonje
Copy link
Member

nelsonje commented Jan 8, 2015

If that doesn't work, according to the FindMPI.cmake source, we should be able to set the include/library paths and flags directly like this:

./configure -- -DMPI_CXX_INCLUDE_PATH=(something)  -DMPI_CXX_LINK_FLAGS=(something)  -DMPI_CXX_COMPILE_FLAGS=(something)  -DMPI_CXX_LIBRARIES=(something) 

Relevant docs from FindMPI.cmake:

# === Variables ===
#
# This module will set the following variables per language in your project,
# where <lang> is one of C, CXX, or Fortran:
#   MPI_<lang>_FOUND           TRUE if FindMPI found MPI flags for <lang>
#   MPI_<lang>_COMPILER        MPI Compiler wrapper for <lang>
#   MPI_<lang>_COMPILE_FLAGS   Compilation flags for MPI programs
#   MPI_<lang>_INCLUDE_PATH    Include path(s) for MPI header
#   MPI_<lang>_LINK_FLAGS      Linking flags for MPI programs
#   MPI_<lang>_LIBRARIES       All libraries to link MPI programs against

and

# === Usage ===
#
# To use this module, simply call FindMPI from a CMakeLists.txt file, or
# run find_package(MPI), then run CMake.  If you are happy with the auto-
# detected configuration for your language, then you're done.  If not, you
# have two options:
#   1. Set MPI_<lang>_COMPILER to the MPI wrapper (mpicc, etc.) of your
#      choice and reconfigure.  FindMPI will attempt to determine all the
#      necessary variables using THAT compiler's compile and link flags.
#   2. If this fails, or if your MPI implementation does not come with
#      a compiler wrapper, then set both MPI_<lang>_LIBRARIES and
#      MPI_<lang>_INCLUDE_PATH.  You may also set any other variables
#      listed above, but these two are required.  This will circumvent
#      autodetection entirely.
# When configuration is successful, MPI_<lang>_COMPILER will be set to the
# compiler wrapper for <lang>, if it was found.  MPI_<lang>_FOUND and other
# variables above will be set if any MPI implementation was found for <lang>,
# regardless of whether a compiler was found.

@nelsonje
Copy link
Member

nelsonje commented Jan 8, 2015

For further reference, when I configure using GCC48 and the Intel MPI, these are what the CMake MPI_* variables discussed above are set to:

MPI_CXX_COMPILER='/sampa/share/intel-cluster-studio/impi_latest/bin64/mpigxx'
MPI_CXX_COMPILE_FLAGS=''
MPI_CXX_INCLUDE_PATH='/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/include'
MPI_CXX_LINK_FLAGS=''
MPI_CXX_LIBRARIES='/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/lib/libmpicxx.so;/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/lib/libmpifort.so;/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/lib/release_mt/libmpi.so;/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/lib/libmpigi.a;/usr/lib64/libdl.so;/usr/lib64/librt.so;/usr/lib64/libpthread.so'

and for C (not sure if these are important for our build):

MPI_C_COMPILER='/sampa/share/intel-cluster-studio/impi_latest/bin64/mpigcc'
MPI_C_COMPILE_FLAGS=''
MPI_C_INCLUDE_PATH='/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/include'
MPI_C_LINK_FLAGS=''
MPI_C_LIBRARIES='/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/lib/libmpifort.so;/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/lib/release_mt/libmpi.so;/sampa/share/intel-cluster-studio/impi/5.0.2.044/intel64/lib/libmpigi.a;/usr/lib64/libdl.so;/usr/lib64/librt.so;/usr/lib64/libpthread.so'

The FindMPI code discovers these by running mpigxx -show; if you can't get CMake to identify these paths automatically, you can run a command like that and translate into CMake's format.

@nelsonje
Copy link
Member

nelsonje commented Jan 8, 2015

I should also point out that in my bash environment I have the environment variables CC=gcc and CXX=g++ set. You can also set these with a single option to our configure script, like this:

./configure --cc=/sampa/share/gcc-4.8.2/rtf/bin/gcc -- -DMPI_C_COMPILER=/sampa/share/intel-cluster-studio/impi_latest/bin64/mpigcc -DMPI_CXX_COMPILER=/sampa/share/intel-cluster-studio/impi_latest/bin64/mpigxx

@rfvander
Copy link
Author

rfvander commented Jan 8, 2015

New issue, configure chokes on -cc:
[rfvander@bar1 grappa]$ ./configure -- --cc=/usr/bin/gcc -DMPI_C_COMPILER=mpigcc -DMPI_CXX_COMPILER=mpigxx
Must specify C compiler (either use '--cc=' flag, or set the environment variables CC & CXX
hint: if the compiler you want is on your PATH, you can do: --cc=$(which gcc)...

@rfvander
Copy link
Author

rfvander commented Jan 8, 2015

Hi Jacob,

Sadly, none of this worked. Configure keeps not wanting to recognize MPI_C(XX)_LIBRARIES, even though I define them (I have already exported CC and CXX, and Cmake does find those):
[rfvander@bar1 grappa]$ export MPI_CXX_LIBRARIES=/opt/intel/tools/impi/5.0.1.035/lib64
[rfvander@bar1 grappa]$ export MPI_C_LIBRARIES=/opt/intel/tools/impi/5.0.1.035/lib64
[rfvander@bar1 grappa]$ \rm -rf build
[rfvander@bar1 grappa]$ ./configure
cmake /lustre/home/rfvander/grappa -G"Unix Makefiles" -DSHMMAX=33554432 -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DBASE_C_COMPILER=gcc -DBASE_CXX_COMPILER=g++ -DCMAKE_BUILD_TYPE=RelWithDebInfo
-- The C compiler identification is GNU 4.8.3
-- The CXX compiler identification is GNU 4.8.3
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Boost found: 1.53.0 -- /usr
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:108 (message):
Could NOT find MPI_C (missing: MPI_C_LIBRARIES)
Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:315 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake/Modules/FindMPI.cmake:587 (find_package_handle_standard_args)
CMakeLists.txt:205 (find_package)

-- Configuring incomplete, errors occurred!
See also "/lustre/home/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeOutput.log".
[

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Thursday, January 08, 2015 12:56 AM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

If that doesn't work, according to the FindMPI.cmake source, we should be able to set the include/library paths and flags directly like this:

./configure -- -DMPI_CXX_INCLUDE_PATH=(something) -DMPI_CXX_LINK_FLAGS=(something) -DMPI_CXX_COMPILE_FLAGS=(something) -DMPI_CXX_LIBRARIES=(something)

Relevant docs from FindMPI.cmake:

=== Variables ===

This module will set the following variables per language in your project,

where is one of C, CXX, or Fortran:

MPI__FOUND TRUE if FindMPI found MPI flags for

MPI__COMPILER MPI Compiler wrapper for

MPI__COMPILE_FLAGS Compilation flags for MPI programs

MPI__INCLUDE_PATH Include path(s) for MPI header

MPI__LINK_FLAGS Linking flags for MPI programs

MPI__LIBRARIES All libraries to link MPI programs against

and

=== Usage ===

To use this module, simply call FindMPI from a CMakeLists.txt file, or

run find_package(MPI), then run CMake. If you are happy with the auto-

detected configuration for your language, then you're done. If not, you

have two options:

1. Set MPI__COMPILER to the MPI wrapper (mpicc, etc.) of your

choice and reconfigure. FindMPI will attempt to determine all the

necessary variables using THAT compiler's compile and link flags.

2. If this fails, or if your MPI implementation does not come with

a compiler wrapper, then set both MPI__LIBRARIES and

MPI__INCLUDE_PATH. You may also set any other variables

listed above, but these two are required. This will circumvent

autodetection entirely.

When configuration is successful, MPI__COMPILER will be set to the

compiler wrapper for , if it was found. MPI__FOUND and other

variables above will be set if any MPI implementation was found for ,

regardless of whether a compiler was found.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-69151893.

@rfvander
Copy link
Author

rfvander commented Jan 8, 2015

And this is what happens if I put things on the command line (mpigcc and mpigxx are in my path):
[rfvander@bar1 grappa]$ ./configure -- -DMPI_C_COMPILER=mpigcc -DMPI_CXX_COMPILER=mpigxx
cmake /lustre/home/rfvander/grappa -G"Unix Makefiles" -DSHMMAX=33554432 -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DBASE_C_COMPILER=gcc -DBASE_CXX_COMPILER=g++ -DCMAKE_BUILD_TYPE=RelWithDebInfo -DMPI_C_COMPILER=mpigcc -DMPI_CXX_COMPILER=mpigxx
-- The C compiler identification is GNU 4.8.3
-- The CXX compiler identification is GNU 4.8.3
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Boost found: 1.53.0 -- /usr
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:108 (message):
Could NOT find MPI_C (missing: MPI_C_LIBRARIES)
Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:315 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake/Modules/FindMPI.cmake:587 (find_package_handle_standard_args)
CMakeLists.txt:205 (find_package)

-- Configuring incomplete, errors occurred!
See also "/lustre/home/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeOutput.log".

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Thursday, January 08, 2015 1:24 AM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

I should also point out that in my bash environment I have the environment variables CC=gcc and CXX=g++ set. You can also set these with a single option to our configure script, like this:

./configure --cc=/sampa/share/gcc-4.8.2/rtf/bin/gcc -- -DMPI_C_COMPILER=/sampa/share/intel-cluster-studio/impi_latest/bin64/mpigcc -DMPI_CXX_COMPILER=/sampa/share/intel-cluster-studio/impi_latest/bin64/mpigxx


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-69154718.

@rfvander
Copy link
Author

Hi Jacob,

I am now trying to build your implementation of synch_p2p using the uts example in the grappa repo as an example. However, uts as described in README-Grappa.md does not build.

  1.   There is no file called Makefile in the uts directory
    
  2.   When I ask to use Makefile.uts, which is present, the grappa target cannot be found
    

Then I tried another example, sort, which doesn’t have a Makefile. Then I looked at isopath, which has a grappa subdirectory with a Makefile. Typing make there produced the following:

[rfvander@bar1 grappa]$ make

Makefile:9: //include.mk: No such file or directory

Makefile:41: //system/Makefile: No such file or directory

Makefile:78: warning: overriding recipe for target `run'

Makefile:75: warning: ignoring old recipe for target `run'

make: *** No rule to make target `//system/Makefile'. Stop.

Perhaps it is time for a little primer how to build a grappa application? Thanks.

Rob

@rfvander
Copy link
Author

Hello Jacob,

While the build problem is now resolved on my research cluster, I am having continued problems with building on my production cluster. It does not have access to the Internet, so I downloaded the third party packages and built using –no-download. I also specify all the compilers in the same way as on my research cluster, but I keep getting error messages. As you can see (I added the definition of environment variables that I set before building), Cmake cannot find MPI_CXX or MPI_CXX_LIBRARIES, even though these variables are explicitly defined. Could you give me an idea how to work around this problem? Ultimately, I want to compare timings, and I won’t be able to do that on our research cluster. Thanks.
BTW, I am little puzzled by the build output that Boost could not be found and that it is downloading that. Probably it’s innocuous, but you may want to change that warning.

Rob

[rfvander@eln4 grappa]$ \rm -rf build/
[rfvander@eln4 grappa]$ ./configure --no-downloads
cmake /panfs/panfs3/users3/rfvander/grappa -G"Unix Makefiles" -DSHMMAX=91239737344 -DNO_DOWNLOADS=true -DCMAKE_C_COMPILER=mpigcc -DCMAKE_CXX_COMPILER=mpigxx -DBASE_C_COMPILER=mpigcc -DBASE_CXX_COMPILER=mpigxx -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBOOST_ROOT=/sampa/share/gcc-4.7.2/src/boost_1_51_0
-- The C compiler identification is GNU 4.4.7
-- The CXX compiler identification is GNU 4.4.7
-- Check for working C compiler: /opt/intel/impi/5.0.1.035/intel64/bin/mpigcc
-- Check for working C compiler: /opt/intel/impi/5.0.1.035/intel64/bin/mpigcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /opt/intel/impi/5.0.1.035/intel64/bin/mpigxx
-- Check for working CXX compiler: /opt/intel/impi/5.0.1.035/intel64/bin/mpigxx -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Boost not found.
!! Will download and build Boost, which may take a while.
-- Found MPI_C: /opt/intel/impi/5.0.1.035/intel64/bin/mpigcc
CMake Error at /opt/crtdc/cmake/3.0.2/share/cmake-3.0/Modules/FindPackageHandleStandardArgs.cmake:136 (message):
Could NOT find MPI_CXX (missing: MPI_CXX_LIBRARIES)
Call Stack (most recent call first):
/opt/crtdc/cmake/3.0.2/share/cmake-3.0/Modules/FindPackageHandleStandardArgs.cmake:343 (_FPHSA_FAILURE_MESSAGE)
/opt/crtdc/cmake/3.0.2/share/cmake-3.0/Modules/FindMPI.cmake:611 (find_package_handle_standard_args)
CMakeLists.txt:205 (find_package)

-- Configuring incomplete, errors occurred!
See also "/panfs/panfs3/users3/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeOutput.log".
[rfvander@eln4 grappa]$ history | grep export | tail -10
879 export MPI_C_COMPILER=mpigcc
952 export CC=mpigcc
953 export CXX=mpigxx
954 export MPI_C_COMPILER=mpigcc
955 export MPI_CXX_COMPILER=mpigxx
993 export CC=mpigcc; export CXX=mpigxx; export MPI_C_COMPILER=mpigcc; export MPI_CXX_COMPILER=mpigxx
1002 h | grep export
1020 export MPI_CXX_LIBRARIES=/opt/intel/impi/5.0.1.035/intel64/lib
1052 export MPI_CXX=mpigxx
1055 history | grep export | tail -10

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Tuesday, January 13, 2015 1:39 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

As for the subject of this ticket:

When we last talked we had two problems:

  1. CMake couldn't find your MPI install. This appeared to have something to do with the way your Lustre shared filesystem was set up.
  2. Once we explicitly specified the MPI dependences, we ran into another problem, where Make couldn't seem to find /usr/lib64/libpthread.so.

It appears that you've made progress on one of both of these? What happened?

I would like to figure out more of what was going wrong with MPI discovery so I can file a bug with the CMake folks.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-69825305.

@rfvander
Copy link
Author

This issue is still open for me, unfortunately. The only grappa codes I have been able to build are integrated in your package, and as such are not a model for what an application developer would do. Could you send me a simple example: a tar with just an example makefile and a source code? Thanks.

Rob

From: Van Der Wijngaart, Rob F
Sent: Wednesday, January 21, 2015 11:36 AM
To: 'uwsampa/grappa'
Subject: Building my own grappa application

Hi Jacob,

I am now trying to build your implementation of synch_p2p using the uts example in the grappa repo as an example. However, uts as described in README-Grappa.md does not build.

  1.   There is no file called Makefile in the uts directory
    
  2.   When I ask to use Makefile.uts, which is present, the grappa target cannot be found
    

Then I tried another example, sort, which doesn’t have a Makefile. Then I looked at isopath, which has a grappa subdirectory with a Makefile. Typing make there produced the following:

[rfvander@bar1 grappa]$ make

Makefile:9: //include.mk: No such file or directory

Makefile:41: //system/Makefile: No such file or directory

Makefile:78: warning: overriding recipe for target `run'

Makefile:75: warning: ignoring old recipe for target `run'

make: *** No rule to make target `//system/Makefile'. Stop.

Perhaps it is time for a little primer how to build a grappa application? Thanks.

Rob

@nelsonje
Copy link
Member

Hi Rob,

We're taking a moment to remove some complexity from our build system before updating the docs with details on adding new code. I'll get back to you shortly.

@rfvander
Copy link
Author

Great, thanks, Jacob. I hope you’re not getting frustrated with all my questions, and hope that the result of all of this will be that Grappa will be easier to use for everybody.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Thursday, January 22, 2015 2:05 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Hi Rob,

We're taking a moment to remove some complexity from our build system before updating the docs with details on adding new code. I'll get back to you shortly.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71108639.

@nelsonje
Copy link
Member

Not at all! It's immensely helpful. I just hope I can make progress fast enough to keep you interested while not neglecting my other responsibilities. :-)

Can we schedule some screen-sharing time to debug the MPI problem?

@rfvander
Copy link
Author

Absolutely! I’ll send an invite if you give me an indication of your availability. Thanks, Jacob.

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Thursday, January 22, 2015 2:53 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Not at all! It's immensely helpful. I just hope I can make progress fast enough to keep you interested while not neglecting my other responsibilities. :-)

Can we schedule some screen-sharing time to debug the MPI problem?


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71116002.

@nelsonje
Copy link
Member

After further debugging, we've determined that this MPI detection error is due to a bug in the Intel mpicc wrapper script---in versions prior to 5.0.2 it doesn't propagate errors from the underlying compiler, which confuses CMake's MPI detection script.

I see three ways to solve this now:

  1. Use a newer version of Intel MPI; version 5.0.2 should work fine. (I was using 5.0.2.044.)

  2. The CMake folks have also recently added code to solve this problem, which should be available in CMake verison 3.2; here's the bug report:
    https://public.kitware.com/Bug/view.php?id=15182
    That version of CMake is still in development; you could potentially try downloading and building from their trunk, but that would be a pain.

  3. Since we want to use GCC with Intel MPI, it ought to work to point CMake at the GCC wrapper scripts directly like this:

CC=gcc CXX=g++ ; ./configure -- -DMPI_C_COMPILER=mpigcc -DMPI_CXX_COMPILER=mpigxx

This works for me when I hack my mpicc script and works for one of the users in the CMake bug report, but it could behave differently on your system if something else is also going on. Note that gcc/g++ here must be at least version 4.7.2.

@rfvander
Copy link
Author

Thanks, Jacob. I could confirm that the proper error propagation does work for MPI version 5.0.2, and not for the version I was using earlier. The difference is in the mpigcc scripts, not mpicc. So I am pointing to the newer MPI now. I’d like to note, though, that we ultimately want to link with the Intel compilers, not GNU.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, January 26, 2015 12:48 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

After further debugging, we've determined that this MPI detection error is due to a bug in the Intel mpicc wrapper script---in versions prior to 5.0.2 it doesn't propagate errors from the underlying compiler, which confuses CMake's MPI detection script.

I see three ways to solve this now:

  1. Use a newer version of Intel MPI; version 5.0.2 should work fine. (I was using 5.0.2.044.)

  2. The CMake folks have also recently added code to solve this problem, which should be available in CMake verison 3.2; here's the bug report:
    https://public.kitware.com/Bug/view.php?id=15182
    That version of CMake is still in development; you could potentially try downloading and building from their trunk, but that would be a pain.

  3. Since we want to use GCC with Intel MPI, it ought to work to point CMake at the GCC wrapper scripts directly like this:

CC=gcc CXX=g++ ; ./configure -- -DMPI_C_COMPILER=mpigcc -DMPI_CXX_COMPILER=mpigxx

This works for me when I hack my mpicc script and works for one of the users in the CMake bug report, but it could behave differently on your system if something else is also going on. Note that gcc/g++ here must be at least version 4.7.2.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71535046.

@nelsonje
Copy link
Member

Great! When you're able to build a run a binary we can close the ticket.

As for using the Intel compiler, I'll track that in #205.

@rfvander
Copy link
Author

Sadly, while configure now breezed through, the build failed. Here is the end of the build output.

Rob

common.copy /panfs/panfs3/users3/rfvander/grappa/build/Make+Release/third-party/lib/libboost_prg_exec_monitor.a
gcc.compile.c++ bin.v2/libs/test/build/gcc-4.4.7/release/link-static/threading-multi/exception_safety.o
gcc.compile.c++ bin.v2/libs/test/build/gcc-4.4.7/release/link-static/threading-multi/interaction_based.o
gcc.compile.c++ bin.v2/libs/test/build/gcc-4.4.7/release/link-static/threading-multi/logged_expectations.o
gcc.archive bin.v2/libs/test/build/gcc-4.4.7/release/link-static/threading-multi/libboost_unit_test_framework.a
common.copy /panfs/panfs3/users3/rfvander/grappa/build/Make+Release/third-party/lib/libboost_unit_test_framework.a
...updated 10706 targets...
[ 30%] No install step for 'third-party-boost'
[ 30%] Completed 'third-party-boost'
[ 30%] Built target third-party-boost
Scanning dependencies of target all-third-party
[ 30%] Built target all-third-party
Scanning dependencies of target graph500-generator
Scanning dependencies of target Communicator
[ 30%] Building C object third-party/graph500-generator/CMakeFiles/graph500-generator.dir/graph_generator.c.o
[ 35%] [ 35%] [ 35%] Building C object third-party/graph500-generator/CMakeFiles/graph500-generator.dir/make_graph.c.o
Building C object third-party/graph500-generator/CMakeFiles/graph500-generator.dir/splittable_mrg.c.o
Building C object third-party/graph500-generator/CMakeFiles/graph500-generator.dir/utils.c.o
[ 35%] Building CXX object system/CMakeFiles/Communicator.dir/Communicator.cpp.o
[ 38%] Building CXX object system/CMakeFiles/Communicator.dir/LocaleSharedMemory.cpp.o
cc1plus: error: unrecognized command line option "-std=c++11"
make[2]: *** [system/CMakeFiles/Communicator.dir/Communicator.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
cc1plus: error: unrecognized command line option "-std=c++11"
make[2]: *** [system/CMakeFiles/Communicator.dir/LocaleSharedMemory.cpp.o] Error 1
make[1]: *** [system/CMakeFiles/Communicator.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Linking C static library libgraph500-generator.a
[ 38%] Built target graph500-generator
make: *** [all] Error 2
[rfvander@eln4 Make+Release]$

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, January 26, 2015 1:51 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Great! When you're able to build a run a binary we can close the ticket.

As for using the Intel compiler, I'll track that in #205#205.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71545776.

@nelsonje
Copy link
Member

Would you verify that your GCC version is >= 4.7.2 with gcc --version?

@rfvander
Copy link
Author

It isn’t, just checked, so I’ll move to 4.9, which is available in the corner of my system.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, January 26, 2015 2:03 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Would you verify that your GCC version is >= 4.7.2 with gcc --version?


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71548067.

@rfvander
Copy link
Author

Sigh. This is what happens when I upgrade to gcc v 4.9 and use the latest MPI compiler:

[rfvander@eln4 grappa]$ ./configure --no-downloads
cmake /panfs/panfs3/users3/rfvander/grappa -G"Unix Makefiles" -DSHMMAX=91239737344 -DNO_DOWNLOADS=true -DCMAKE_C_COMPILER=mpigcc -DCMAKE_CXX_COMPILER=mpigxx -DBASE_C_COMPILER=mpigcc -DBASE_CXX_COMPILER=mpigxx -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBOOST_ROOT=/sampa/share/gcc-4.7.2/src/boost_1_51_0
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
-- Check for working C compiler: /opt/intel/impi/5.0.2.044/intel64/bin/mpigcc
-- Check for working C compiler: /opt/intel/impi/5.0.2.044/intel64/bin/mpigcc -- broken
CMake Error at /opt/crtdc/cmake/3.0.2/share/cmake-3.0/Modules/CMakeTestCCompiler.cmake:61 (message):
The C compiler "/opt/intel/impi/5.0.2.044/intel64/bin/mpigcc" is not able
to compile a simple test program.
It fails with the following output:
Change Dir: /panfs/panfs3/users3/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/gmake" "cmTryCompileExec2839858443/fast"
/usr/bin/gmake -f CMakeFiles/cmTryCompileExec2839858443.dir/build.make
CMakeFiles/cmTryCompileExec2839858443.dir/build
gmake[1]: Entering directory
`/panfs/panfs3/users3/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeTmp'

/opt/crtdc/cmake/3.0.2/bin/cmake -E cmake_progress_report
/panfs/panfs3/users3/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeTmp/CMakeFiles
1
Building C object
CMakeFiles/cmTryCompileExec2839858443.dir/testCCompiler.c.o
/opt/intel/impi/5.0.2.044/intel64/bin/mpigcc -o
CMakeFiles/cmTryCompileExec2839858443.dir/testCCompiler.c.o -c
/panfs/panfs3/users3/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeTmp/testCCompiler.c

/opt/crtdc/gcc/gcc-4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/cc1:
error while loading shared libraries: libmpc.so.3: cannot open shared
object file: No such file or directory
gmake[1]: *** [CMakeFiles/cmTryCompileExec2839858443.dir/testCCompiler.c.o]
Error 1
gmake[1]: Leaving directory
`/panfs/panfs3/users3/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeTmp'

gmake: *** [cmTryCompileExec2839858443/fast] Error 2

CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:19 (project)

-- Configuring incomplete, errors occurred!
See also "/panfs/panfs3/users3/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeOutput.log".
See also "/panfs/panfs3/users3/rfvander/grappa/build/Make+Release/CMakeFiles/CMakeError.log".

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, January 26, 2015 2:03 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Would you verify that your GCC version is >= 4.7.2 with gcc --version?


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71548067.

@nelsonje
Copy link
Member

Would you verify that you can build a simple plain C program with GCC 4.9, and a MPI program with mpigcc and GCC 4.9? The library it's complaining about is part of GCC, so if mpigcc can't find GCC's library include paths we would expect this sort of error.

@nelsonje
Copy link
Member

Oh, and it looks like this git clone doesn't have the SHMMAX fix---you should do a pull to get the latest bits.

@rfvander
Copy link
Author

Right, that’s the problem. I’ve poked around, but nothing compiles with this version of gcc on our system. I’m asking the admins to install a new version, or patch up the one we have.

Rob

[rfvander@eln4 Transpose]$ more test.c
int main(int argc, char**argv){
int i;
}
[rfvander@eln4 Transpose]$ gcc test.c
/opt/crtdc/gcc/gcc-4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/cc1: error while loading shared libraries: libmpc.so.3: cannot open shared object file: No such file or directory

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, January 26, 2015 3:20 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Would you verify that you can build a simple plain C program with GCC 4.9, and a MPI program with mpigcc and GCC 4.9? The library it's complaining about is part of GCC, so if mpigcc can't find GCC's library include paths we would expect this sort of error.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71559690.

@rfvander
Copy link
Author

Will do, thanks.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, January 26, 2015 11:36 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Oh, and it looks like this git clone doesn't have the SHMMAX fix---you should do a pull to get the latest bits.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71601727.

@rfvander
Copy link
Author

OK, Jacob, progress on my production cluster. It turns out that not all compiler dependences were set. I won’t bore you with the details, but suffice it to say that after correcting that, and after pulling the new bits, I could configure and make grappa (of course, I also needed to do the no-downloads hack). But I could not build hello_world. Error log attached.

[rfvander@eln4 Make+Release]$ make demo-hello_world 2> error.log
[ 10%] Built target third-party-gflags
[ 20%] Built target third-party-boost
[ 30%] Built target third-party-glog
[ 30%] Built target all-third-party
[ 35%] Built target graph500-generator
[ 97%] Built target Grappa
Linking CXX executable hello_world.exe
[rfvander@eln4 Make+Release]$ wc error.log
1656 10181 222660 error.log
[rfvander@eln4 Make+Release]$ grep -10 world.exe error.log
/panfs/panfs3/users3/rfvander/grappa/system/tasks/TaskingScheduler.hpp:471: undefined reference to google::LogMessageFatal::LogMessageFatal(char const*, int)' ../../system/libGrappa.a(GlobalMemory.cpp.o): In functionLinear':
/panfs/panfs3/users3/rfvander/grappa/system/Addressing.hpp:172: undefined reference to google::LogMessageFatal::LogMessageFatal(char const*, int, google::CheckOpString const&)' /panfs/panfs3/users3/rfvander/grappa/system/Addressing.hpp:172: undefined reference togoogle::LogMessageFatal::~LogMessageFatal()'
/panfs/panfs3/users3/rfvander/grappa/system/Addressing.hpp:173: undefined reference to google::LogMessageFatal::LogMessageFatal(char const*, int, google::CheckOpString const&)' /panfs/panfs3/users3/rfvander/grappa/system/Addressing.hpp:173: undefined reference togoogle::LogMessageFatal::~LogMessageFatal()'
../../system/libGrappa.a(GlobalMemory.cpp.o): In function Allocator': /panfs/panfs3/users3/rfvander/grappa/system/Allocator.hpp:172: undefined reference togoogle::LogMessageFatal::LogMessageFatal(char const_, int)'
/panfs/panfs3/users3/rfvander/grappa/system/Allocator.hpp:172: undefined reference to `google::LogMessageFatal::~LogMessageFatal()'
collect2: error: ld returned 1 exit status
make[3]: *_* [applications/demos/hello_world.exe] Error 1
make[2]: *** [applications/demos/CMakeFiles/demo-hello_world.dir/all] Error 2
make[1]: *** [applications/demos/CMakeFiles/demo-hello_world.dir/rule] Error 2
make: *** [demo-hello_world] Error 2
From: Jacob Nelson [mailto:notifications@github.com]
Sent: Monday, January 26, 2015 11:36 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Oh, and it looks like this git clone doesn't have the SHMMAX fix---you should do a pull to get the latest bits.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71601727.

@nelsonje
Copy link
Member

Great! Simon and I just saw this error in a VM he was building. It went away when we blew away the build directory and rebuilt (with no -j flag specified). I'm trying to reproduce it again.

Would you try running like this and send me the output?

make demo-hello_world VERBOSE=1

And if convenient would you tar up your build directory and send it to me too? I'll point you at a FTP server you can drop it on.

@rfvander
Copy link
Author

Um, OK, you may be excited, but I just want to build the darn thing ☺. Anyway, I’ll do the whole thing again and stuff everything you need at that FTP site. And after that I’ll do the non-parallel make so I can finally build and run some Grappa code! I’m at home with slow internet now, so won’t be moving the gobs of data to your FTP site until tomorrow at work.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Tuesday, January 27, 2015 5:40 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Great! Simon and I just saw this error in a VM he was building. It went away when we blew away the build directory and rebuilt (with no -j flag specified). I'm trying to reproduce it again.

Would you try running like this and send me the output?

make demo-hello_world VERBOSE=1

And if convenient would you tar up your build directory and send it to me too? I'll point you at a FTP server you can drop it on.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71766053.

@nelsonje
Copy link
Member

Hey, I'm just excited that I could reproduce this one. You can skip sending the build directory since I've been able to repeat it now.

@rfvander
Copy link
Author

OK, Jacob, back at my desk now. So I should not do anything at this point?

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Wednesday, January 28, 2015 10:05 AM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Hey, I'm just excited that I could reproduce this one. You can skip sending the build directory since I've been able to repeat it now.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71883822.

@nelsonje
Copy link
Member

Yes, hold off for a bit---the serial build doesn't avoid the problem. I'll let you know when my debugging yields something.

@rfvander
Copy link
Author

OK, I’ll immediately start doing nothing, then.

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Wednesday, January 28, 2015 10:42 AM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Yes, hold off for a bit---the serial build doesn't avoid the problem. I'll let you know when my debugging yields something.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-71890407.

@simonkahan
Copy link
Member

What did you do differently that repeated it?

On Wed, Jan 28, 2015 at 10:42 AM, Jacob Nelson notifications@github.com
wrote:

Yes, hold off for a bit---the serial build doesn't avoid the problem. I'll
let you know when my debugging yields something.


Reply to this email directly or view it on GitHub
#198 (comment).

@nelsonje
Copy link
Member

(replying belatedly to Simon: I built Boost from scratch rather than pointing at an already-installed version. It turns out our build system made different assumptions in that case which are no longer true.)

@nelsonje
Copy link
Member

Okay, Rob, I just merged some more build system changes into master. If you pull and rebuild you should not get the error you saw.

I'll have a makefile solution shortly as well.

@rfvander
Copy link
Author

Terrific, Jacob. I really appreciate your efforts and patience. I’ll let you know how I fare.

Rob

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Friday, January 30, 2015 2:55 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Okay, Rob, I just merged some more build system changes into master. If you pull and rebuild you should not get the error you saw.

I'll have a makefile solution shortly as well.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-72284978.

@rfvander
Copy link
Author

The sweet smell of success! Have a good weekend, Jacob.

Rob

[rfvander@eln4 Make+Release]$ make demo-hello_world
[ 10%] Built target third-party-gflags
[ 20%] Built target third-party-boost
[ 30%] Built target third-party-glog
[ 30%] Built target all-third-party
[ 35%] Built target graph500-generator
[ 97%] Built target Grappa
Scanning dependencies of target demo-hello_world
[100%] Building CXX object applications/demos/CMakeFiles/demo-hello_world.dir/hello_world/hello_world.cpp.o
Linking CXX executable hello_world.exe
[100%] Built target demo-hello_world

From: Jacob Nelson [mailto:notifications@github.com]
Sent: Friday, January 30, 2015 2:55 PM
To: uwsampa/grappa
Cc: Van Der Wijngaart, Rob F
Subject: Re: [grappa] Build problem with MPI library (#198)

Okay, Rob, I just merged some more build system changes into master. If you pull and rebuild you should not get the error you saw.

I'll have a makefile solution shortly as well.


Reply to this email directly or view it on GitHubhttps://github.com//issues/198#issuecomment-72284978.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants