HPX build failing with HWLOC errors on POWER8 with hwloc 1.8

[khuck]

Expected Behavior

As of 10 hours ago, the build was fine, but one of the most recent pull requests has broken the build.

Actual Behavior

HPX build fails with:

/home/users/khuck/buildbot/slaves/phylanx/ppc64le-clang5-release/build/tools/buildbot/src/hpx/src/runtime/threads/topology.cpp:521:42: error: use of undeclared identifier 'HWLOC_MEMBIND_BYNODESET'
                    1, nodeset, &policy, HWLOC_MEMBIND_BYNODESET);
                                         ^
/home/users/khuck/buildbot/slaves/phylanx/ppc64le-clang5-release/build/tools/buildbot/src/hpx/src/runtime/threads/topology.cpp:1236:55: error: use of undeclared identifier 'HWLOC_MEMBIND_BYNODESET'
            (hwloc_membind_policy_t)(policy), flags | HWLOC_MEMBIND_BYNODESET);
                                                      ^
/home/users/khuck/buildbot/slaves/phylanx/ppc64le-clang5-release/build/tools/buildbot/src/hpx/src/runtime/threads/topology.cpp:1246:42: error: use of undeclared identifier 'HWLOC_MEMBIND_BYNODESET'
            topo, addr, len, ns, policy, HWLOC_MEMBIND_BYNODESET);
                                         ^
/home/users/khuck/buildbot/slaves/phylanx/ppc64le-clang5-release/build/tools/buildbot/src/hpx/src/runtime/threads/topology.cpp:1279:47: error: use of undeclared identifier 'HWLOC_MEMBIND_BYNODESET'
                topo, addr, len, ns, &policy, HWLOC_MEMBIND_BYNODESET) == -1)
                                              ^
4 errors generated.
make[3]: *** [src/CMakeFiles/hpx.dir/runtime/threads/topology.cpp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [src/CMakeFiles/hpx.dir/all] Error 2
make[1]: *** [CMakeFiles/core.dir/rule] Error 2
make: *** [core] Error 2

Steps to Reproduce the Problem

... Please be as specific as possible while describing how to reproduce your problem.

1. cmake -DCMAKE_BUILD_TYPE=Release -DBOOST_ROOT=/home/users/khuck/buildbot/slaves/phylanx/ppc64le-clang5-release/build/tools/buildbot/build-centaur-ppc64le-Linux-clang/boost-1.65.0 -DTCMALLOC_ROOT=/usr/local/packages/gperftools/2.5 -DHPX_WITH_MALLOC=tcmalloc -DHWLOC_ROOT= -DCMAKE_INSTALL_PREFIX=. -DHPX_WITH_THREAD_IDLE_RATES=ON -DHPX_WITH_PARCELPORT_MPI=ON -DHPX_WITH_PARCEL_COALESCING=OFF -DHPX_WITH_TOOLS=ON -DHPX_WITH_APEX=TRUE -DAPEX_WITH_ACTIVEHARMONY=TRUE -DACTIVEHARMONY_ROOT=/usr/local/packages/activeharmony/4.6.0-ppc64le -DAPEX_WITH_OTF2=TRUE -DOTF2_ROOT=/usr/local/packages/otf2/2.0-ppc64le -DAPEX_WITH_PAPI=TRUE -DPAPI_ROOT=/usr/local/packages/papi/5.5.1 -DHPX_WITH_APEX_NO_UPDATE=TRUE -DHPX_WITH_APEX_TAG=develop -DHWLOC_ROOT=/usr/local/packages/hwloc/1.8 -DHPX_WITH_RDTSCP=OFF -DHPX_WITH_DEPRECATION_WARNINGS=Off -DHPX_WITH_MAX_CPU_COUNT=160 /home/users/khuck/buildbot/slaves/phylanx/ppc64le-clang5-release/build/tools/buildbot/src/hpx
2. make

Specifications

... Please describe your environment

• HPX Version: current master
• Platform (compiler, OS): Clang 5.0 on POWER8 linux, hwloc 1.8

[msimberg]

Thanks for reporting. So this is because of #3445. I'd be okay with reverting the hwloc related changes there. Alternatively, we could raise our minimum supported version to 1.11 (or something in between 1.11 and 1.8, whatever works). @khuck is it possible for you to try 1.11 instead?

[khuck]

Yeah, I'll try with 1.11. Keep in mind that many HPC systems could have older OpenMPI / hwloc installed and won't have the option to upgrade...

[hkaiser]

@msimberg can't we #ifdef this code depending on the HWLOC version?

[khuck]

builds fine with 1.11. Should I close this, or keep it open until the code handles (or rejects during configuration) 1.8?

[msimberg]

@hkaiser, yeah, we can do that. I hoped to avoid that but at least with ifdefs we'll have an easy time bumping the minimum version when the time comes. @khuck leave it it open for now. I'll open a PR tomorrow.

[biddisco]

@hkaiser @msimberg The problem is that versions of hwloc were released with the wrong version numbers in the version file, and so #ifdefing them doesn't work in some cases. There simply isn't a fix that will work for everyone other than insisting on a very recent version of hwloc. (As I hinted in the PR a few days ago).

[biddisco]

NB. I had a number of problems with hwloc and I can't remember which symbols were the ones that caused trouble, these HWLOC_MEMBIND_BYNODESET might not be the problem ones. I need to check my logs ...

[msimberg]

@biddisco If you can dig up the problematic versions that would be nice. I thought it was within the 1.11.X releases that they messed up the versions? But maybe they introduced HWLOC_MEMBIND_BYNODESET in one of those releases... We can always use an overly pessimistic version check (HWLOC_MEMBIND_BYNODESET could be available but we still use the old API).

[hkaiser]

@khuck please verify this has been fixed.