Intel OPA support?

[gshamov]

Hi,

Is Intel OmniPath supported for the recent GA 5.7?
I can build it, but one of the tests keeps failing.

--
Grigory Shamov
University of Manitoba

[jeffhammond]

Use ARMCI_NETWORK=MPI-PR with Intel Omni Path.

[gshamov]

Somehow didn't help.

[jeffhammond]

Please provide full details if you want help debugging.

[gshamov]

Hi Jeff,

Sorry about the delay. Looks like it was a problem with our OpeMPI.: the test perf_strided was having error "Exhausted 1048576 MQ irecv ... PSM2_MQ_RECVREQS_MAX=1048576". I was attempting to compile GA with Intel compilers 2016.4.258 and OpenMPI 2.1.1 . I did the following configure options:

ARMCI_NETWORK=MPI-PR ./configure --enable-ofi --with-mpi-pr CC=icc FC=ifort

When I have tried OpenMPI 3.1.2 also with Intel compilers, and the same configure flags the problem went away and all the tests passed.

[jeffhammond]

@gshamov Ah, okay, I have seen that one before with NWChem. Try PSM2_MEMORY=large.

It seems the GASNet folks have encountered this error and solution before (https://bitbucket.org/berkeleylab/gasnet/pull-requests/73/fix-for-psm-conduit-puts-reporting/activity#comment-30348715).

[gshamov]

Thanks!
Another question: can GA binaries understand more than one fabric? Like. both OPA and IB, if configured with multiple --enable- flags?

[jeffhammond]

@gshamov No, GA does not support that, but MPI and OFI do, so if you want a binary that runs over multiple fabrics, configure GA to use MPI-PR or ARMCI-MPI and run with an MPI library that supports IB+OPA. Intel MPI does that automatically, Open-MPI can be built to do it, and both MPICH and Open-MPI can do it via OFI if OFI is configured properly.

[jeffhammond]

@gshamov Note that this is great feature request but it is a lot of work and is unlikely to be implemented, particularly since it is much easier to improve GA support for MPI and punt the problem to MPI.

[gshamov]

Thanks!