Build failure with Intel compiler (main branch of open mpi) #11133

MamziB · 2022-11-30T20:00:10Z

I am trying to build the latest main branch using the intel compiler. But it is throwing the following error:

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_components_open.c(184): error #188: enumerated type mixed with another type
                      ret = PMIX_ERR_BAD_PARAM;

Thoughts/comments? Configure option follows:

module load intel/2022.1.2 compiler/2022.1.0 mkl/2022.0.2


-C --enable-debug --with-ompi-param-check --enable-picky --prefix=xx --with-ucx=xx--with-verbs --enable-mpi1-compatibility --without-xpmem --with-libevent=/usr --with-slurm --with-pmix=internal --with-hwloc=internal CC=icc CXX=icpc F77=ifort FC=ifort

Adding the pmix commit that is getting used:

commit 250004266bc046c6303c8531ababdff4e1237525 (HEAD)
Author: Samuel K. Gutierrez <samuel@lanl.gov>
Date:   Mon Nov 21 13:00:34 2022 -0700

    Fix perf_tools build, silence warnings.

    Signed-off-by: Samuel K. Gutierrez <samuel@lanl.gov>

The text was updated successfully, but these errors were encountered:

MamziB · 2022-11-30T20:06:09Z

Full error log:

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_framework.c(73): error #188: enumerated type mixed with another type
          flags |= PMIX_MCA_BASE_REGISTER_STATIC_ONLY;
                ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_framework.c(136): error #188: enumerated type mixed with another type
      framework->framework_flags |= PMIX_MCA_BASE_FRAMEWORK_FLAG_REGISTERED;
                                 ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_framework.c(161): error #188: enumerated type mixed with another type
          flags |= PMIX_MCA_BASE_OPEN_FIND_COMPONENTS;
                ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_framework.c(164): error #188: enumerated type mixed with another type
              flags |= PMIX_MCA_BASE_OPEN_STATIC_ONLY;
                    ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_framework.c(180): error #188: enumerated type mixed with another type
          framework->framework_flags |= PMIX_MCA_BASE_FRAMEWORK_FLAG_OPEN;
                                     ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_framework.c(232): error #188: enumerated type mixed with another type
      framework->framework_flags &= ~(PMIX_MCA_BASE_FRAMEWORK_FLAG_REGISTERED
                                 ^

compilation aborted for /global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_framework.c (code 2)
make[4]: *** [Makefile:830: pmix_mca_base_framework.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_var.c(674): error #188: enumerated type mixed with another type
      var->mbv_flags &= ~PMIX_MCA_BASE_VAR_FLAG_VALID;
                     ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_var.c(1271): error #188: enumerated type mixed with another type
      var->mbv_flags |= PMIX_MCA_BASE_VAR_FLAG_VALID;
                     ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_var.c(1323): error #188: enumerated type mixed with another type
          var_flags |= PMIX_MCA_BASE_VAR_FLAG_DEPRECATED;
                    ^

compilation aborted for /global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_var.c (code 2)
make[4]: *** [Makefile:830: pmix_mca_base_var.lo] Error 1
/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_components_open.c(141): error #188: enumerated type mixed with another type
                  ret = PMIX_ERROR;
                      ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_components_open.c(156): error #188: enumerated type mixed with another type
                      ret = PMIX_ERROR;
                          ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_components_open.c(168): error #188: enumerated type mixed with another type
                      ret = PMIX_ERROR;
                          ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_components_open.c(184): error #188: enumerated type mixed with another type
                      ret = PMIX_ERR_BAD_PARAM;
                          ^

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_components_open.c(193): error #188: enumerated type mixed with another type
                      ret = PMIX_ERR_OUT_OF_RESOURCE;
                          ^

compilation aborted for /global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_components_open.c (code 2)
make[4]: *** [Makefile:830: pmix_mca_base_components_open.lo] Error 1
remark #11074: Inlining inhibited by limit max-size
remark #11076: To get full report use -qopt-report=4 -qopt-report-phase ipo
make[4]: Leaving directory '/global/scratch/users/mbayatpour/build-x86/ompi-github-debug-intel/3rd-party/openpmix/src/mca/base'
make[3]: *** [Makefile:1323: install-recursive] Error 1
make[3]: Leaving directory '/global/scratch/users/mbayatpour/build-x86/ompi-github-debug-intel/3rd-party/openpmix/src'
make[2]: *** [Makefile:848: install-recursive] Error 1
make[2]: Leaving directory '/global/scratch/users/mbayatpour/build-x86/ompi-github-debug-intel/3rd-party/openpmix'
make[1]: *** [Makefile:1400: install-recursive] Error 1
make[1]: Leaving directory '/global/scratch/users/mbayatpour/build-x86/ompi-github-debug-intel/3rd-party'
make: *** [Makefile:1525: install-recursive] Error 1

rhc54 · 2022-11-30T20:10:13Z

Turn off the "picky" behavior or you'll drive yourself mad with that compiler.

MamziB · 2022-11-30T20:30:39Z

@rhc54 still no luck. I removed --enable-picky:

  $ /global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/configure -C --enable-debug --with-ompi-param-check --prefix=/global/scratch/users/mbayatpour/build-x86/ompi-github-debug-intel --with-ucx=/global/scratch/users/mbayatpour/build-x86/ucx-no-debug-intel --with-verbs --enable-mpi1-compatibility --without-xpmem --with-libevent=/usr --with-slurm --with-pmix=internal --with-hwloc=internal CC=icc CXX=icpc F77=ifort FC=ifort

rhc54 · 2022-11-30T20:33:09Z

You might have to check and see what OMPI is adding itself and tell it to stop adding some of the flags. Intel has what I'd consider a bug in its compiler that causes it to not consider enum variables to be int.

MamziB · 2022-11-30T20:45:36Z

Thanks, @rhc54. But I am still puzzled about how to fix this. Any other thoughts? Doesn't this look like a code issue? I was able to compile open mpi with intel compiler a few months ago. But now it fails.

MamziB · 2022-11-30T20:48:55Z

FYI, I found another issue Snaipe/Criterion#76 with a similar failure. This issue was resolved by changing the code.

hppritcha · 2022-11-30T20:50:37Z

if you're building pmix/prrte as submodules of ompi you're always building these with the --enable-picky option.

Built pmix standalone and with --disable-picky and the the compiler "error" will vanish.

Are you using intel classic or intel oneapi compilers?

hppritcha · 2022-11-30T20:52:00Z

never mind didn't see you're full config line.

rhc54 · 2022-11-30T20:54:36Z

I believe you just have to add --disable-picky to the configure line

Doesn't this look like a code issue?

Not really - according to the standard, enum equates to int. I somewhat guess what Intel is attempting to do, but I'm not convinced of the value nor the correctness of it. Chasing compiler problems is an exhausting exercise that strains my "retired" status. If someone would like to chase it and provide a patch, I'm happy to take a look at it.

hppritcha · 2022-11-30T21:01:29Z

no pmix has a hardwired thing in its configury. It all flows from this logic:

    # A hint to tell us if we are working with a build from Git or a tarball.
    # Helpful when preparing diagnostic output.
    if test -e $PMIX_TOP_SRCDIR/.git; then
        AC_DEFINE_UNQUOTED([PMIX_GIT_REPO_BUILD], ["1"],
            [If built from a git repo])
        pmix_git_repo_build=yes
    fi

this overrides the --disable-picky from the ompi config line at present.

rhc54 · 2022-11-30T21:02:37Z

Ah - okay, that is easily fixed

MamziB · 2022-11-30T21:07:31Z

@hppritcha which file can I change to avoid this override? I prefer not to build my own openmpix. I don't want to deal with an extra separate build.

rhc54 · 2022-11-30T21:09:40Z

Just adding --disable-devel-check to the configure line should resolve what @hppritcha reported

MamziB · 2022-11-30T22:17:49Z

@rhc54 @hppritcha Thanks everyone - looks like adding --disable-devel-check is being helpful.

rhc54 · 2022-12-01T15:09:30Z

Just so you understand why I say this is a compiler bug, take this one instance:

/global/scratch/users/mbayatpour/workspace4/ompi-git-repo/ompi/3rd-party/openpmix/src/mca/base/pmix_mca_base_framework.c(73): error #188: enumerated type mixed with another type
          flags |= PMIX_MCA_BASE_REGISTER_STATIC_ONLY;

Now look at the code:

enum pmix_mca_base_register_flag_t {
    PMIX_MCA_BASE_REGISTER_DEFAULT = 0,
    /** Register all components (ignore selection MCA variables) */
    PMIX_MCA_BASE_REGISTER_ALL = 1,
    /** Do not register DSO components */
    PMIX_MCA_BASE_REGISTER_STATIC_ONLY = 2
};
typedef enum pmix_mca_base_register_flag_t pmix_mca_base_register_flag_t;

and where it is used:

int pmix_mca_base_framework_register(struct pmix_mca_base_framework_t *framework,
                                     pmix_mca_base_register_flag_t flags)
<space>
        flags |= PMIX_MCA_BASE_REGISTER_STATIC_ONLY;

So you can see that the "flags" variable and the value it is being set to both match in declared type. Yet the compiler complains anyway. Best we have been able to decipher is that the compiler gets confused by the setting of the enum value to a specific integer, and thus thinks that it is of type "int" - and therefore conflicts in type with the variable since Intel decided that "enum" doesn't default to base type "int".

The only way we have found to "fix" it is either to remove all enum's from the code, or to remove all value settings from within enum definitions. The latter doesn't match the C standard, which is annoying but tolerable. The option we would likely pursue is the former, but that takes effort that is better used on other things.

I'm sure we will eventually succumb and remove the enum's - it is just incredibly frustrating to have these compiler "quirks" force projects to waste effort, and to lose the debugger feature of showing you the enum name instead of some integer value.

hppritcha · 2022-12-01T15:27:27Z

well I would not get too concerned here as the intel classic compilers, which @MamziB isusing are going into maintenance mode (i forgot the exact Intel terminiology) starting this coming January as I understand it. This problem may go away by using the Intel OneAPI compilers.

hppritcha · 2022-12-01T15:27:28Z

well I would not get too concerned here as the intel classic compilers, which @MamziB isusing are going into maintenance mode (i forgot the exact Intel terminiology) starting this coming January as I understand it. This problem may go away by using the Intel OneAPI compilers.

MamziB · 2022-12-02T20:18:19Z

@hppritcha I double-checked with our system administrator. Classic compilers are too old and we don’t use them. So all these compiler complaints are coming from oneAPI:
icc --version
icc (ICC) 2021.6.0 20220226

rhc54 · 2022-12-02T20:20:30Z

Perhaps someone should report the bug to Intel? You can use the example I posted above - it's pretty basic and clearly wrong.

rhc54 · 2022-12-02T20:26:16Z

@hppritcha I double-checked with our system administrator. Classic compilers are too old and we don’t use them. So all these compiler complaints are coming from oneAPI:
icc --version icc (ICC) 2021.6.0 20220226

No, those are the classic compilers. The oneAPI compiler names are CC=icx CXX=icpx.

MamziB · 2022-12-02T20:45:13Z

@rhc54 Good point. I did not know this. I tried icx and icpx and now openmpi compiles.

jsquyres mentioned this issue Dec 2, 2022

OpenMPI Does not configure with Intel ICX Compilers, ICC planned obselete 2023Q3 #11142

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build failure with Intel compiler (main branch of open mpi) #11133

Build failure with Intel compiler (main branch of open mpi) #11133

MamziB commented Nov 30, 2022 •

edited

MamziB commented Nov 30, 2022

rhc54 commented Nov 30, 2022

MamziB commented Nov 30, 2022

rhc54 commented Nov 30, 2022

MamziB commented Nov 30, 2022

MamziB commented Nov 30, 2022

hppritcha commented Nov 30, 2022

hppritcha commented Nov 30, 2022

rhc54 commented Nov 30, 2022

hppritcha commented Nov 30, 2022

rhc54 commented Nov 30, 2022

MamziB commented Nov 30, 2022

rhc54 commented Nov 30, 2022

MamziB commented Nov 30, 2022

rhc54 commented Dec 1, 2022

hppritcha commented Dec 1, 2022

hppritcha commented Dec 1, 2022

MamziB commented Dec 2, 2022

rhc54 commented Dec 2, 2022

rhc54 commented Dec 2, 2022 •

edited

MamziB commented Dec 2, 2022

Build failure with Intel compiler (main branch of open mpi) #11133

Build failure with Intel compiler (main branch of open mpi) #11133

Comments

MamziB commented Nov 30, 2022 • edited

MamziB commented Nov 30, 2022

rhc54 commented Nov 30, 2022

MamziB commented Nov 30, 2022

rhc54 commented Nov 30, 2022

MamziB commented Nov 30, 2022

MamziB commented Nov 30, 2022

hppritcha commented Nov 30, 2022

hppritcha commented Nov 30, 2022

rhc54 commented Nov 30, 2022

hppritcha commented Nov 30, 2022

rhc54 commented Nov 30, 2022

MamziB commented Nov 30, 2022

rhc54 commented Nov 30, 2022

MamziB commented Nov 30, 2022

rhc54 commented Dec 1, 2022

hppritcha commented Dec 1, 2022

hppritcha commented Dec 1, 2022

MamziB commented Dec 2, 2022

rhc54 commented Dec 2, 2022

rhc54 commented Dec 2, 2022 • edited

MamziB commented Dec 2, 2022

MamziB commented Nov 30, 2022 •

edited

rhc54 commented Dec 2, 2022 •

edited