dyld: malformed mach-o: load commands size (33848) > 32768

[Fanxiaotsing]

All the examples can not run on macos catalina10.15.4 with 9.2.0 installed,can anyone fix this problem?use the "otool -l"can get
otool -l /Applications/deal.II.app/Contents/Resources/Libraries/lib/libdeal_II.g.9.2.0.dylib
/Applications/deal.II.app/Contents/Resources/Libraries/lib/libdeal_II.g.9.2.0.dylib:
Mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
0xfeedfacf 16777223 3 0x00 6 196 38368 0x00118085
Load command 0
cmd LC_SEGMENT_64
cmdsize 632
segname __TEXT
vmaddr 0x0000000000000000
vmsize 0x0000000009087000
fileoff 0
filesize 151547904
maxprot 0x00000005
initprot 0x00000005
nsects 7
flags 0x0
Section
sectname __text
segname __TEXT
addr 0x0000000000031260
size 0x0000000006b90d33
offset 201312
align 2^4 (16)
reloff 0
nreloc 0
flags 0x80000400
reserved1 0
reserved2 0

@drwells edit: trimmed for brevity

[drwells]

This looks like a problem with our mac bundle. Let me see if I can replicate it.

[drwells]

@luca-heltai I can reproduce this issue on my mac (a week-old mac mini running Catalina). Let me know when you are available to take a look.

[luca-heltai]

Unfortunately, I cannot reproduce on my mac. We could take a look at this today?

[Fanxiaotsing]

So that means I have to build the library from the bottom?

[drwells]

Unfortunately, until we get this fixed that will be necessary.

[drwells]

@luca-heltai I have time to fix this but I don't know how to build or test the mac bundle. Could you help me by pointing me to instructions on how to rebuild the mac bundle with spack on my machine?

[luca-heltai]

I've tried to fix this in the past week, with no luck. The way I proceed is the following:

1. I create the directory

/Applications/deal.II.app/Contents/Resources/

2. in that directory, I clone the spack repository. Now you have

/Applications/deal.II.app/Contents/Resources/spack

Call this SPACK_ROOT

3. somewhere in the path I download and expand a version of gfortran (i.e.,

$SPACK_ROOT/gcc-9.2

for example, from http://prdownloads.sourceforge.net/hpc/gfortran-9.2-bin.tar.gz?download)

4. I run the https://gist.github.com/luca-heltai/d52952a8c6334a1c5374bd96fb165c08#file-relocate_libraries-py script in that directory, so that gfortran is correctly relocated (with absolute paths)

5. I make sure spack knows about gfortran

$SPACK_ROOT/bin/spack compiler add --scope site $SPACK_ROOT/gcc-9.2

6. I modify

$SPACK_ROOT/etc/spack/compilers.yaml

so that clang and gcc-9.2 are both present as cxx and gfortran compilers:

compilers:
- compiler:
    spec: clang@11.0.3-apple
    paths:
      cc: /usr/bin/clang
      cxx: /usr/bin/clang++
      f77: /Applications/deal.II.app/Contents/Resources/spack/gcc-9.2/bin/gfortran
      fc: /Applications/deal.II.app/Contents/Resources/spack/gcc-9.2/bin/gfortran
    flags: {}
    operating_system: catalina
    target: x86_64
    modules: []
    environment: {unset: []}
    extra_rpaths: []

7. Now run

$SPACK_ROOT/bin/spack install dealii@9.2

If this succeeds, you should be able to run deal.II examples. After this, we can make a bundle by filling the rest of theapp directory.

[luca-heltai]

I was able to do this for the package, but it did not work correctly for some of the libraries. I ran relocate_libraries.py in the $SPACK_ROOT/opt/ directory, and this produced a linking library. However, it seems like on some machines the resulting libraries have too many load commands, and this fails (not everywhere, and not always...).

[luca-heltai]

I think I was able (FINALLY) to fix the package. I'm uploading my package right now. I could not set all libraries with their absolute path, so the solution is either to use modules (module load deal.II), or to enable the view, in which case DYLD_LIBRARY_PATH=/Applications/deal.II.app/Contents/Resources/Libraries/lib is set.

Notice that this is deprecated and should not happen, but I have no way to circumvent the load size. We load too many libraries, and spack creates paths that are too long...

I have uploaded a new package on the deal.II 9.2.0 release page. @drwells can you check if this works for you?

[drwells]

It looks like it was not signed correctly - I get the following error:

"deal.II.app" cannot be opened because the developer cannot be verified.

[luca-heltai]

You should "ctrl" click. This would open a dialog where you are asked if you trust me. I believe you should say yes here. ;)

[drwells]

Oh, duh (you can tell that I don't normally use a mac).

The symbolic links in, e.g., deal.II.app/Contents/Resources/Libraries/share/deal.II/examples/step-1 are broken: it looks like the examples weren't installed.

I also see a warning

ld: warning: directory not found for option `-L/Applications/deal.II.app/Contents/Resources/spack/opt/spack/darwin-catalina-x86_64/clang-11.0.3-apple/zlib-1.2.11-rqiqrujgg5aemhk7eq....

Otherwise things appear to be in order. I can run step-40 in parallel just fine.

[Fanxiaotsing]

Hello,everyone!I have built deal.ii9.2.0 on my macbook using Spack.Here is my steps:
1、spack install gcc@9.3.0
2、spack load gcc
spack compiler find
3、since on MacOs ,gcc conflicts with cmake and python,we cannot just use' spack install dealii%gcc';instead we can use 'spack install deallii%clang',before that the fortran compiler's path must be assigned( gfortan under the gcc9.3.0/bin )
so the compilers.yaml looks like
compilers:

• compiler:
  spec: clang@11.0.3-apple
  paths:
  cc: /usr/bin/clang
  cxx: /usr/bin/clang++
  f77: /Users/spack-develop/opt/spack/darwin-catalina-x86_64/clang-11.0.3-apple/gcc-9.3.0-th2gg6gtbyz5ipoh5joauzedtwau6goi/bin/gfortran
  fc: /Users/spack-develop/opt/spack/darwin-catalina-x86_64/clang-11.0.3-apple/gcc-9.3.0-th2gg6gtbyz5ipoh5joauzedtwau6goi/bin/gfortran
  flags: {}
  operating_system: catalina
  target: x86_64
  modules: []
  environment: {unset: []}
  extra_rpaths: []
  4、Some libraries(for me ,they are libamd.2.dylib,libcamd.2.dylib,libccolamd.2.dylib,libcholmod.3.dylib,libcolamd.2.dylib,libmetis.dylib,libumfpack.5.dylib,libsuitesparseconfig.5.dylib) can not be loaded during the make phase,this can be cured by symbolic linking（though it is not elegant).
  for example
  ln -s /Users/spack-develop/.../suitesparse.../lib/libumfpack.5.7.9.dylib libumfpack.5.dylib
  each time copy those links to your own dir
  5、it works.

[Fanxiaotsing]

@luca-heltai @drwells @tamiko Thanks a lot!

[jppelteret]

4、Some libraries can not be loaded during the make phase,this can be cured by symbolic linking（though it is not elegant).

This can be fixed by using either filesystem views or modules. They both allow you to modify the environment variables such that all libraries are detected and consistent with how the dependencies are build. You can also see the deal.II wiki for a little more detail on how you'd use them to build deal.II.

Personally, I used to use views because they were implemented in spack before modules and seemed really easy and intuitive to use. But last time I installed deal.II on a cluster and my workstation I tried modules. I get the feeling the modules are more robust (since loading the modules will make spack modify the environment, rather than you needing to do it yourself). So I would now recommend the modules approach, even though the syntax is a little weird and it takes a little time to understand how to set them up.

[luca-heltai]

Notice that the deal.II package is already set up with modules on by default:

export DEAL_II_CONF_SILENT=ON
export DEAL_II_ENABLE_VIEW=OFF # Use modules
. /Applications/deal.II.app/Contents/MacOS/dealii.conf

will allow you to simply write module load dealii before running/compiling/etc.

[luca-heltai]

This should avoid all problems @Fanxiaotsing is observing with the libraries

[luca-heltai]

Can we close this? Alberto Salvadori also confirmed that the package is working for him now.

[drwells]

This works for me too (with the not-important exceptions listed above) so yes.