delocate.delocating.DelocationError: .dylibs/libgfortran.5.dylib already exists

[letmaik]

This worked in the past but suddenly I get an error when running delocate-path on a folder where multiple executables rely on the same dependency (I'm assuming that's what the error is about): "delocate.delocating.DelocationError: .dylibs/libgfortran.5.dylib already exists". Am I doing something wrong?

[letmaik]

@matthew-brett What can I do to help you analyze the problem?

[matthew-brett]

Yes, sorry - can you make me some way of replicating the problem?

…

On Thu, Jun 6, 2019 at 9:40 PM Maik Riechert ***@***.***> wrote: @matthew-brett <https://github.com/matthew-brett> What can I do to help you analyze the problem? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#53?email_source=notifications&email_token=AAAQQHHYSFPDDNKVDJNBYYTPZFY2JA5CNFSM4HN5LD4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXEDBAQ#issuecomment-499658882>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAQQHBDRUOTB5X2BMCIBRLPZFY2JANCNFSM4HN5LD4A> .

[letmaik]

@matthew-brett I couldn't reproduce it locally yet, only on CI weirdly. Will investigate further. But if there's any verbose logging etc I can enable, let me know.

[matthew-brett]

Can you point me to the error on CI log?

[letmaik]

https://dev.azure.com/wrf-cmake/WRF/_build/results?buildId=428 See the failed macOS build and click on the "Delocate" step.

[letmaik]

I dumped out delocate-listdeps in case it's helpful:

+ delocate-listdeps --all --depending build/install/main
/usr/lib/libSystem.B.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/lib/libz.1.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/libgcc_s.1.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/libgfortran.5.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/libquadmath.0.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/local/Cellar/jasper/2.0.16_1/lib/libjasper.5.0.0.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/local/Cellar/jpeg/9c/lib/libjpeg.9.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/local/Cellar/libpng/1.6.37/lib/libpng16.16.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/local/Cellar/netcdf/4.6.3_1/lib/libnetcdf.15.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
/usr/local/Cellar/netcdf/4.6.3_1/lib/libnetcdff.6.1.1.dylib:
    build/install/main/ideal.exe
    build/install/main/ideal_fire.exe
    build/install/main/ideal_heldsuarez.exe
    build/install/main/ideal_scm_xy.exe
    build/install/main/ideal_tropical_cyclone.exe
    build/install/main/real.exe
    build/install/main/wrf.exe
+ delocate-path build/install/main
Traceback (most recent call last):
  File "/usr/local/bin/delocate-path", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/delocate/cmd/delocate_path.py", line 39, in main
    delocate_path(path, lib_path, lib_filt_func)
  File "/usr/local/lib/python3.7/site-packages/delocate/delocating.py", line 289, in delocate_path
    return copy_recurse(lib_path, copy_filt_func, copied)
  File "/usr/local/lib/python3.7/site-packages/delocate/delocating.py", line 145, in copy_recurse
    _copy_required(lib_path, copy_filt_func, copied_libs)
  File "/usr/local/lib/python3.7/site-packages/delocate/delocating.py", line 230, in _copy_required
    raise DelocationError(out_path + ' already exists')
delocate.delocating.DelocationError: build/install/main/.dylibs/libgfortran.5.dylib already exists

[letmaik]

OK, I found the issue.

libgfortran.5.dylib was a direct dependency of the executables listed above, but also an indirect dependency via libnetcdff.6.1.1.dylib (which is not listed in delocate-listdeps). The catch was that the direct dependency came from /usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/libgfortran.5.dylib and the indirect one from /usr/local/Cellar/gcc@9/9.1.0/lib/gcc/9/libgfortran.5.dylib. delocate then couldn't handle the situation of different absolute paths with the same filename. I changed my build to use the same gfortran for everything which solved the issue.

There are two things that should be improved I think:

• Show indirect dependencies in delocate-listdeps.
• Improve the error message for this situation and include the absolute paths of the two conflicting dylibs.

[HexDecimal]

PR #94 should have improved situations with indirect dependencies. Can you test your wheel with the latest version of delocate?

delocate-listdeps also shows indirect dependencies since #118 was merged.