Segmentation fault: 11

[ZiyanZhu1994]

Hi,

I updated my Mac OS system several days ago. When I run the PyCUTEst today, it returns some warnings and a Segmentation fault:

ld: warning: directory not found for option '-L/usr/local/gfortran/lib/'
ld: warning: object file (RANGE.o) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (EXTER.o) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (ELFUN.o) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (GROUP.o) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cdh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cdimen.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cdimsh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cdimsj.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cfn.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cidh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ccfg.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ccfsg.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ccifg.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ccifsg.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cgr.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cgrdh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(chprod.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cjprod.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cofg.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(csetup.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(csgr.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(csgrsh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(uhprod.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(uofg.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cish.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(connames.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(creport.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(csh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cvartype.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(probname.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(udh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(udimsh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ugrdh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ugrsh.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ureport.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(usetup.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(ush.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(uvartype.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(varnames.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(fortran_ops.o)) was built for newer OSX version (10.14) than being linked (10.9)
ld: warning: object file (/Users/ziyan/Documents/Github/cutest/cutest/objects/mac64.osx.gfo/double/libcutest.a(cutest.o)) was built for newer OSX version (10.14) than being linked (10.9)
Segmentation fault: 11

Do you know why it happens and how to fix it? Many thanks!

[jfowkes]

Hmm... could you reinstall CUTEst then clear out the PyCUTEst cache and try again? I suspect that this is an issue with incompatible compiler versions.

[ZiyanZhu1994]

I have reinstalled the CUTEst using the brew reinstall. When I run the PyCUTEst in the terminal, the same warnings are returned, but it can work. However, when I try to run PyCUTEst in Jupyter notebook or other IDEs, the kernel will be died and Segmentation fault error will occur.

[jfowkes]

Could you please install the latest version of Command Line Tools using xcode-select --install as described here: http://railsapps.github.io/xcode-command-line-tools.html? Then clear out the PyCUTEst cache and try again.

[ZiyanZhu1994]

I reinstalled the Xcode and clear out the cache. It still returns these warnings. Below is the version of gcc:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
I think it is the latest version. The version of gfortran is:
GNU Fortran (Homebrew GCC 9.2.0) 9.2.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[jfowkes]

Yes that's exactly the same as I have. You're using Anaconda's python right? Have you also updated Anaconda to the latest version for you newly upgraded Mac OS system?

[ZiyanZhu1994]

Yes, I have updated the anaconda. Below is the information of my anaconda:
_
active environment : base
active env location : //anaconda3
shell level : 1
user config file : /Users/ziyan/.condarc
populated config files : /Users/ziyan/.condarc
conda version : 4.7.11
conda-build version : 3.18.9
python version : 3.7.4.final.0
virtual packages :
base environment : //anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : //anaconda3/pkgs
/Users/ziyan/.conda/pkgs
envs directories : //anaconda3/envs
/Users/ziyan/.conda/envs
platform : osx-64
user-agent : conda/4.7.11 requests/2.22.0 CPython/3.7.4 Darwin/18.7.0 OSX/10.14.6
UID:GID : 501:20
netrc file : None
offline mode : False
_

By the way, my Mac OS version is Mojave 10.14.6.

[jfowkes]

Okay so I installed Anaconda and have been able to reproduce and fix your error. The issue is that for some stupid reason Anaconda sets the environment variable MACOSX_DEPLOYMENT_TARGET=10.9 by default (yes that's right, MacOS Mavericks!). So somewhere you have to set MACOSX_DEPLOYMENT_TARGET=10.14 either by doing export MACOSX_DEPLOYMENT_TARGET=10.14 in the terminal or in .bashrc. If you're using an IDE, you'll have to set it in the appropriate place for setting environment variables in your IDE (see the documentation for your IDE). There may well be a way of changing this stupid default somewhere in Anaconda itself, but I'm not familiar enough with Anaconda to know where that would be and I haven't been able to find anything.

[ZiyanZhu1994]

Thanks! The linking warnings are fixed! There is still one warning:
ld: warning: directory not found for option '-L/usr/local/gfortran/lib/'
and Segmentation fault:11. I have tried to reinstall the gcc and cutest using brew, but the error is still there.

[jfowkes]

You can ignore the option '-L/usr/local/gfortran/lib/' warning, that's just an additional gfortran directory we added that we should probably remove. That's strange, it works for me:

What is your gfortran --version? Are you using the one provided by Anaconda? I am not.

[ZiyanZhu1994]

Yes, it is strange. It used to work well before I updating MacOS from 10.14.5 to 10.14.6. gfortran was installed using brew along with gcc. Here is the version of it:
GNU Fortran (Homebrew GCC 9.2.0) 9.2.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Also, I run
cd ~/Documents/pycutest_cache/pycutest_cache_holder/ROSENBR/
rm -rf build/
DISTUTILS_DEBUG=1 python setup.py build
it returns

options (after parsing config files):
options (after parsing command line):
option dict for 'build' command:
{}
running build
Distribution.get_command_obj(): creating 'build' command object
running build_ext
Distribution.get_command_obj(): creating 'build_ext' command object
building '_pycutestitf' extension
creating build
creating build/temp.macosx-10.9-x86_64-3.7
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I//anaconda3/include -arch x86_64 -I//anaconda3/include -arch x86_64 -DLINUX -I//anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy -I//anaconda3/include/python3.7m -c cutestitf.c -o build/temp.macosx-10.9-x86_64-3.7/cutestitf.o
creating build/lib.macosx-10.9-x86_64-3.7
gcc -bundle -undefined dynamic_lookup -L//anaconda3/lib -arch x86_64 -L//anaconda3/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.9-x86_64-3.7/cutestitf.o RANGE.o EXTER.o ELFUN.o GROUP.o /usr/local/opt/cutest/libexec/objects/mac64.osx.gfo/double/libcutest.a -L/usr/local/Cellar/gcc/9.2.0/lib/gcc/9/ -L/usr/local/gfortran/lib/ -lgfortran -o build/lib.macosx-10.9-x86_64-3.7/_pycutestitf.cpython-37m-darwin.so -Wl,-no_compact_unwind
ld: warning: directory not found for option '-L/usr/local/gfortran/lib/'

[jfowkes]

Yes that's the same gcc version as I have. According to your output, you're still building for MacOS 10.9 though, you should be seeing creating build/temp.macosx-10.14-x86_64-3.7, could you try export MACOSX_DEPLOYMENT_TARGET=10.14 and then DISTUTILS_DEBUG=1 python setup.py build and post the output? I can then compare it to mine and we should be able to figure out what's going on.

[ZiyanZhu1994]

I have tried export MACOSX_DEPLOYMENT_TARGET=10.14 in the terminal. It still uses MacOS 10.9. However, the issue is solved by creating another python environment using conda, where only PyCUTEst is installed. I guess there might be come conflicts between some packages in anaconda and PyCUTEst.

[jfowkes]

Yes I think that's probably the case, I was discussing this issue with one of the Anaconda developers yesterday and they told me that the environments target 10.9 for compatibility and anything else is only supported on a best-effort basis as they're largely a compile it for you distribution (unlike homebrew).

[ladsjohn]

Curious how you "solved by creating another python environment using conda, where only PyCUTEst is installed".
I tried:
$ conda create -n hansolo
$ conda activate hansolo

I'm still hitting a Seg Fault:11 when trying to run:
problem = pycutest.import_problem('ARGLALE', sifParams={'N':100, 'M':200})
print(problem)

[jfowkes]

The issue here is that Anaconda targets a very old version of MacOS when compiling and you need to tell it to target your current MacOS version (which should be the default but Anaconda developers don't want to change this). You can do this by setting export MACOSX_DEPLOYMENT_TARGET=10.14 (or whatever your current MacOS version is, 10.14=Mojave, 10.15=Catalina, etc) in the terminal before running Anaconda python. You may need to create a new python environment using conda where only PyCUTEst is installed in order for it to work.

In general I would discourage you from using Anaconda with PyCUTEst as it is a compile for you distribution and support for compiling things is experimental as the Anaconda developers made very clear when I raised this issue with them. My recommendation would be to install python via homebrew and use that instead.

[ladsjohn]

I had previously tried setting MACOSX_DEPLOYMENT_TARGET=10.15 in my environment to no avail. On your recommendation; I did have success by not using Anaconda. Many thanks!