Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

openMPI doesn't seem to work correctly on ami-c62a25b2 (ubuntu-12.04-x86_64 (EBS) in eu-west-1) #215

Closed
lefevrer opened this Issue · 8 comments

3 participants

@lefevrer

On ami-c62a25b2, openMPI doesn't seem to work correctly.
Code to produce the unexpected behavior (simple Hello World program in MPI):

#include <stdio.h> /* printf and BUFSIZ defined there */
#include <stdlib.h> /* exit defined there */
#include <mpi.h> /* all MPI-2 functions defined there */

int main(argc, argv)
        int argc;
        char *argv[];
        {
        int rank, size, length;
        char name[BUFSIZ];

        MPI_Init(&argc, &argv);
        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
        MPI_Comm_size(MPI_COMM_WORLD, &size);
        MPI_Get_processor_name(name, &length);

        printf("%s: hello world from process %d of %d\n", name, rank, size);

        MPI_Finalize();

        exit(0);
}

Steps to produce the unexpected behavior:
1. launch two instances with ami-c62a25b2
2. save the code above as "helloworldmpi.c" in /home/sgeadmin
3. run the following lines:

sgeadmin@master:~$ mpicc helloworldmpi.c -o helloworldmpi
sgeadmin@master:~$ mpirun -n 2 -host master,node001 ./helloworldmpi

The output is:

master: hello world from process 0 of 1
node001: hello world from process 0 of 1

but should be:

master: hello world from process 0 of 2
node001: hello world from process 1 of 2

I don't know if the problem above is also present on other ami or other region, but it works correctly on ami-ab447bdf (ubuntu-11.10-x86_64 (EBS) in eu-west-1).

@lefevrer

I did some more tests:

  • helloworldmpi compiled on ami-ab447bdf always works correctly, even on ami-c62a25b2.
  • helloworldmpi compiled ami-c62a25b2 never works correctly.

So, the problem seems to be at the compilation step on ami-c62a25b2 (ubuntu-12.04-x86_64 (EBS) in eu-west-1).

@lefevrer

Here are some outputs on ami-c62a25b2 (ubuntu-12.04-x86_64 (EBS) in eu-west-1):

root@master:~# mpirun -V
mpirun (Open MPI) 1.4.3

Report bugs to http://www.open-mpi.org/community/help/
root@master:~# mpicc -v
mpicc for MPICH2 version 1.4.1
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.6.3-1ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --enable-objc-gc --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) 

So, it seems that mpirun is well linked to openMPI but mpicc is linked to MPICH2. Can you correct that ?

@jtriley
Owner

@lefevrer Thanks for reporting and yes I plan to correct this and republish all of the 12.04 AMIs along with a new 12.04 HVM AMI soon.

@lefevrer

Ok! Thanks for your reply.

@lefevrer lefevrer closed this
@jtriley jtriley reopened this
@jtriley
Owner

@lefevrer This has been fixed. I deregistered all of the old 12.04 AMIs and published the latest version with the MPI fix and a couple other minor fixes to all AWS regions. The new 12.04 AMIs also have julialang included.

@jtriley jtriley closed this
@jtriley
Owner

@lefevrer You'll need to update the AMIs in your config with the latest 12.04 AMI ids. You can get them from:

$ starcluster -r eu-west-1 listpublic
@lefevrer

Thanks a lot !

@nitroamos

I'm able to replicate this problem with:
[1] ami-80bedfb0 us-west-2 starcluster-base-ubuntu-13.04-x86_64-hvm (HVM-EBS)

(! 614)-> mpirun -V
mpirun (Open MPI) 1.4.5

Report bugs to http://www.open-mpi.org/community/help/

(! 615)-> mpicc -v
mpicc for MPICH2 version 1.4.1
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.7.3-1ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs --enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.7 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --with-system-zlib --enable-objc-gc --with-cloog --enable-cloog-backend=ppl --disable-cloog-version-check --disable-ppl-version-check --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-1ubuntu1) 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.