Potential issue when using PARMETIS column permutation in v8.2.1

[v-dobrev]

I noticed this during testing of mfem-4.7 with SuperLU_DIST v8.2.1 -- one of the mfem examples that uses SuperLU_DIST, examples/ex11p.cpp hangs when run with:

mpirun -np 4 ./ex11p --superlu -no-vis

This seems to happen in both CPU and GPU builds.

If the option superlu::PARMETIS used in ex11p.cpp is changed to superlu::METIS_AT_PLUS_A, the issue goes away.

Also, when using SuperLU_DIST v8.1.2, there is no issue. I have not tried v9.0.0.

The easiest way to reproduce this is probably using Spack -- basically build mfem with superlu-dist and enable the mfem build testing with --test=root:

./bin/spack install --fresh --test=root mfem+superlu-dist ^superlu-dist@8.2.1

This install should stall when running ex11p with the superlu-dist option -- just cancel the Spack install at that point with ctrl+c. After the failure, one can go inside the failed build directory and run the failing example (using the mpirun command above) in mfem's examples directory.

[liuyangzhuan]

@v-dobrev There is indeed a bug in the latest superlu_dist code when parmetis is being used. I've fixed it in this commit:
250e7c4
Can you give it a try with the latest superlu_dist code and see if there is the hanging issue? @xiaoyeli

[v-dobrev]

Hi @liuyangzhuan,

I just ran the MFEM (v4.7) tests in a build with SuperLU_DIST (master branch) and there were no issues. It looks like the issue is fixed. Thanks!