Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

'omp.h' file not found on Platform("aarch64", "macos") #18

Closed
amontoison opened this issue Nov 8, 2022 · 28 comments
Closed

'omp.h' file not found on Platform("aarch64", "macos") #18

amontoison opened this issue Nov 8, 2022 · 28 comments

Comments

@amontoison
Copy link
Member 

 Compiling guess_topology  		guess_topology.cxx:13:10: fatal error: 'omp.h' file not found
#include <omp.h>
         ^~~~~~~
1 error generated.
 => Disabling optimization guess_topology.cxx:13:10: fatal error: 'omp.h' file not found
#include <omp.h>
         ^~~~~~~
1 error generated.
make[2]: *** [/workspace/srcdir/GALAHAD/src/ssids/makemaster:310: /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/libgalahad_spral.a(guess_topology.o)] Error 1
make[1]: *** [/workspace/srcdir/GALAHAD/src/makedefs/intermediaries:560: make_ssids] Error 2
make: *** [/workspace/srcdir/GALAHAD/src/makemaster:187: all_standalone_double] Error 2
@dpo
Copy link
Collaborator

dpo commented Nov 8, 2022

What version of gcc?

@amontoison
Copy link
Member Author

It was gcc 11.1.

@amontoison
Copy link
Member Author

https://stackoverflow.com/a/33401294

@nimgould
Copy link
Contributor

nimgould commented Nov 8, 2022 •
edited
Loading

The SOMP = -fopenmp compiler flag should provide all the relevant include data here. Could you please go to the $GALAHAD/src/ssids directory, and try

make -f (architecture)

where (architecture) is whatever name you have in $GALAHAD/makefiles/ (e.g., mac64.osx.gfo). This will tell us what flags are being passed

@amontoison
Copy link
Member Author

amontoison commented Nov 8, 2022 •
edited
Loading

I have an error when I try make -f binarybuilder.bb.fc:

sandbox:${WORKSPACE}/srcdir/GALAHAD/src/ssids # make -f binarybuilder.bb.fc
make: binarybuilder.bb.fc: No such file or directory
make: *** No rule to make target 'binarybuilder.bb.fc'.  Stop.

sandbox:${WORKSPACE}/srcdir/GALAHAD/src/ssids # ls
C                        config.h                 fkeep.F90                gpu_subtree_no_cuda.f90  solve.cu
LICENCE                  contrib.f90              gpu                      inform.f90               ssids.f90
NumericSubtree.cxx       contrib.h                gpu_alloc.f90            ldlt_app.cxx             ssidss.data
README                   contrib_free.f90         gpu_datatypes.f90        ldlt_nopiv.cxx           ssidss.f90
SymbolicSubtree.cxx      cpu                      gpu_dense_factor.f90     ldlt_tpp.cxx             subtree.f90
ThreadStats.cxx          cpu_iface.f90            gpu_factor.f90           makemaster               syrk.cu
akeep.f90                cpu_solve.f90            gpu_interfaces.f90       profile.cxx              updata_ssids
anal.f90                 cpu_subtree.f90          gpu_smalloc.f90          profile.hxx              wrappers.cxx
assemble.cu              datatypes.f90            gpu_solve.f90            profile_iface.f90
cholesky.cxx             dense_factor.cu          gpu_subtree.f90          reorder.cu
sandbox:${WORKSPACE}/srcdir/GALAHAD/src/ssids #

@nimgould
Copy link
Contributor

nimgould commented Nov 8, 2022 via email

@amontoison
Copy link
Member Author 

sandbox:${WORKSPACE}/srcdir/GALAHAD/src/ssids # make -f $GALAHAD/makefiles/binarybuilder.bb.fc
 Compiling guess_topology  		mkdir -p /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/hw_topology
sed -f /workspace/srcdir/GALAHAD/seds/have_hwloc_def.sed ../spral/config.h > /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/config1.h
sed -f /workspace/srcdir/GALAHAD/seds/have_getcpu_undef.sed /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/config1.h > /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/config.h
cp  ../spral/compat.hxx /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/
cp  ../spral/guess_topology.cxx /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/
cp  ../spral/guess_topology.hxx /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/hw_topology/
cp  ../spral/hwloc_wrapper.hxx /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/hw_topology/
cd /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double ; c++ -I. -o guess_topology.o -c -std=c++11 -fPIC -O2   -fopenmp \
          guess_topology.cxx \
                 || ( printf ' %-26s' "=> Disabling optimization " ; \
                    c++ -I. -o guess_topology.o -c -std=c++11 -fPIC -O0  -fopenmp \
                      guess_topology.cxx )
guess_topology.cxx:13:10: fatal error: 'omp.h' file not found
#include <omp.h>
         ^~~~~~~
1 error generated.
 => Disabling optimization guess_topology.cxx:13:10: fatal error: 'omp.h' file not found
#include <omp.h>
         ^~~~~~~
1 error generated.
make: *** [/workspace/srcdir/GALAHAD/src/ssids/makemaster:310: /workspace/srcdir/GALAHAD/objects/binarybuilder.bb.fc/double/libgalahad_spral.a(guess_topology.o)] Error 1

@nimgould
Copy link
Contributor

nimgould commented Nov 8, 2022 via email

@amontoison
Copy link
Member Author

amontoison commented Nov 8, 2022 •
edited
Loading

Yes, cc and c++ are alias for C and C++ compilers.
It's gcc and g++ for Linux / Windows and Clang / Clang++ on Mac and FreeBSD.

@dpo
Copy link
Collaborator

dpo commented Nov 8, 2022

could you try gcc/gfortran for macOS?

@amontoison
Copy link
Member Author

amontoison commented Nov 8, 2022 •
edited
Loading

Yes, I restart the compilation.
I just checked and we use gcc/gfortran for SPRAL:
https://github.com/JuliaPackaging/Yggdrasil/blob/master/S/SPRAL/build_tarballs.jl#L16-L19

@jfowkes
Do you test the compilation of SPRAL on Mac with CI?
It seems that modifications are needed for Clang / Clang++ compilers (native compilers on Mac / FreeBSD).

@amontoison
Copy link
Member Author

The compilation of GALAHAD on Mac works with gcc/gfortran.

@dpo
Copy link
Collaborator

dpo commented Nov 9, 2022

I would suggest that we forget about clang for now, and first stabilize everything with gcc/gfortran on every platform of interest.

@nimgould
Copy link
Contributor

nimgould commented Nov 9, 2022

I believe that Alexis mentioned the work-around here. If
c++ is from clang, one needs SOMP = '-fopenmp=libomp'
in the compiler.binarybuilder.bb.fc file in archdefs, which
is copied into GALAHAD's makefiles/binarybuilder.bb.fc
on installation. Give that a try ... but I agree with Dominique that
we should concentrate on gcc for the time being.

@jfowkes
Copy link
Collaborator

jfowkes commented Nov 9, 2022

@amontoison SPRAL is only tested on Linux GCC in the CI but I have successfully compiled it with Homebrew GCC on Mac.

I believe it would take quite a bit of nontrivial work to make SPRAL work with Clang given how complex it is.

@nimgould
Copy link
Contributor

nimgould commented Nov 9, 2022

I tried flang/clang, and the flag

SOMP = -fopenmp=libomp -I/usr/lib/llvm-14/lib/clang/14.0.0/include

did the trick here ... but as you see it needs to know where the system header is. The alternative

-I/usr/lib/gcc/x86_64-linux-gnu/11/include/omp.h

also worked, and is marginally more portable.

Unfortunately, a following ssids compile

clang++ -I. -o NumericSubtree.o -c -std=c++11 -fPIC -O -fopenmp=libomp -I/usr/lib/llvm-14/lib/clang/14.0.0/include NumericSubtree.cxx

produces a wave of errors, typically
./ssids/cpu/kernels/assemble.hxx:212:42: error: variable 'add_a_blk_sz' must have explicitly specified data sharing attributes
add_a_block(iblk, std::min(iblk+add_a_blk_sz,snode.num_a), node, aval, scaling);

As I don't know c++, I have no idea what this means, and I can't find any compiler flag to disable it (if that is even possible). This does not happen with icx which is also built on top of LLVM, so I suspect it is just clang being too fussy.

@jfowkes
Copy link
Collaborator

jfowkes commented Nov 9, 2022 via email

@nimgould
Copy link
Contributor

nimgould commented Nov 9, 2022

Very interesting. In the block

  #pragma omp taskgroup
  for(int iblk=0; iblk<snode.num_a; iblk+=add_a_blk_sz) {
     #pragma omp task default(none) \
        firstprivate(iblk) \
        shared(snode, node, aval, scaling, ldl)
     add_a_block(iblk, std::min(iblk+add_a_blk_sz,snode.num_a), node, aval, scaling);
  }

what precisely would you remove? I'm willing to try, but need hand holding

@jfowkes
Copy link
Collaborator

jfowkes commented Nov 9, 2022

If my understanding is correct then removing the default(none) after #pragma omp task should fix it.

@nimgould
Copy link
Contributor

nimgould commented Nov 9, 2022

I'll try tomorrow once I "get back" from Edinburgh

@amontoison
Copy link
Member Author

Ok, so the issue is in spral and not in GALAHAD. We could move the issue to the GitHub SPRAL repository directly.
I agree with Dominique, we should compile GALAHAD with gcc/gfortran on all relevant platforms.

@jfowkes I did two CI scripts for a C++ code a few months ago, you can easily adapt them for SPRAL:
https://github.com/bbopt/nomad/blob/master/.github/workflows/ci.yml
https://github.com/bbopt/nomad/blob/master/.cirrus.yml

@jfowkes
Copy link
Collaborator

jfowkes commented Nov 10, 2022

Thanks @amontoison, I've created a SPRAL pull request to test with Clang/Flang:
ralna/spral#97
Flang doesn't seem to be working though, any suggestions? We should discuss there.

@nimgould
Copy link
Contributor

Good call, Jari. removing the three default (none)s fixed the compiler errors, and the sls exhaustive test now runs ... and ssids gives the right answer. I should add that this is with flang/clang as compiled from https://github.com/flang-compiler/llvm.git . Next I need to make sure that these changes don't ruin ssids under other compilers. Progress on an otherwise hopeless day!

@nimgould
Copy link
Contributor

Oh, I should add that I needed to set the galahad variables
OPENMP = -fopenmp -lstdc++ -lhwloc
SOMP = -fopenmp=libomp -I/usr/lib/llvm-14/lib/clang/14.0.0/include
and the other is of course not portable (but presumably will be when flang is finally
released as part of clang

@nimgould
Copy link
Contributor

OK, the updates also work for gcc and icc. icx fails, but I think it did before. Jari, are you planning to make these changes in spral as well?

@jfowkes
Copy link
Collaborator

jfowkes commented Nov 11, 2022

@nimgould yes I will make these changes to SPRAL, could you point me to the three default(none) statements you removed?

@nimgould
Copy link
Contributor

Yes, in assemble.hxx (as per $GALAHAD/src/ssids/cpu/kernel/), lines 209, 281 and 389

@jfowkes
Copy link
Collaborator

jfowkes commented Nov 11, 2022

@nimgould done, fixed in SPRAL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dpo @jfowkes @nimgould @amontoison