Problem with MXM and MPICH builds

[jsquyres]

From Martin Cuma on the mailing list (http://lists.openfabrics.org/pipermail/libfabric-users/2016-December/000145.html):

---

Now, I have another potential quirk with libfabric. I configured it to add Mellanox MXM, as:

$ ../../../srcdir/libfabric/1.4.0/configure --prefix=/uufs/chpc.utah.edu/sys/installdir/libfabric/1.4.0 --enable-mxm=/uufs/chpc.utah.edu/sys/installdir/hpcx/v1.7.405-gcc-OFED-1.5.4.1-redhat7.2-x86_64/mxm

All built fine, the libfabric tests work, but, when building MPICH with the ch4:ofi device, the make was complaining about missing /libmxm.la. After some hunting, I have located the source of this in /uufs/chpc.utah.edu/sys/installdir/libfabric/1.4.0/lib/libfabric.la:

dependency_libs=' -L/uufs/chpc.utah.edu/sys/installdir/hpcx/v1.7.405-gcc-OFED-1.5.4.1-redhat7.2-x86_64/mxm/lib -lrdmacm -lnl-3 -lnl-route-3 -lpsm_infinipath  /libmxm.la -L/usr/lib64 -L/usr/lib -lm -lz -libverbs -lbfd -liberty -lrt -lpthread -ldl'

It looks like the libfabric build has not correctly expanded the full path to the libmxm.la.

Is this something I did incorrectly with the libfabric build, or something that needs to be fixed in the libfabric itself?

[jsquyres]

@shefty git blame says that you're the main culprit behind the MXM provider. Can you have a look at this user question?

[shefty]

I did the merge for mxm, but that was it. @alexander-sannikov is showing as the last committer prior to it being removed. Comparing the psm2 and mxm providers, I don't see this equivalent in mxm Makefile.include:

src_libfabric_la_DEPENDENCIES += libpsmx2.la

[jsquyres]

@shefty Will the MXM provider exist in libfabric 1.5?

[shefty]

No - the code has already been stripped out of the repo. A mlx provider has replaced it.

[jsquyres]

Ok. I think Martin already has a workaround, so let's go ahead and close this.