PyStan 2.18.0.0 compilation failure on macOS in Anaconda environment.

[justinbois]

Summary:

PyStan 2.18.0.0 installed in Anaconda with Python 3.7 on macOS has a compiler error.

Description:

I have tested this on multiple Macs, running 10.12, 10.13, and 10.14, with Anaconda installations using Python 3.6 and Python 3.7. Compilation of any Stan model (a simple one shown below) fails with the error show below. This is not an issue with PyStan 2.17.1.0.

Reproducible Steps:

Upon a fresh installation of Anaconda with Python 3.7, I ran the following code.

import pystan

stan_model_code="""
data {
  real a;
}
generated quantities {
  real b = beta_rng(2.0, 2.0);
}
"""

sm = pystan.StanModel(model_code=stan_model_code)

Current Output:

The current output. Knowing what is the current behavior is useful.

INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_8b588fdcf67e45da3e1122393e447758 NOW.
---------------------------------------------------------------------------
DistutilsExecError                        Traceback (most recent call last)
~/anaconda3/lib/python3.7/distutils/unixccompiler.py in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
    117             self.spawn(compiler_so + cc_args + [src, '-o', obj] +
--> 118                        extra_postargs)
    119         except DistutilsExecError as msg:

~/anaconda3/lib/python3.7/distutils/ccompiler.py in spawn(self, cmd)
    908     def spawn(self, cmd):
--> 909         spawn(cmd, dry_run=self.dry_run)
    910 

~/anaconda3/lib/python3.7/distutils/spawn.py in spawn(cmd, search_path, verbose, dry_run)
     35     if os.name == 'posix':
---> 36         _spawn_posix(cmd, search_path, dry_run=dry_run)
     37     elif os.name == 'nt':

~/anaconda3/lib/python3.7/distutils/spawn.py in _spawn_posix(cmd, search_path, verbose, dry_run)
    158                           "command %r failed with exit status %d"
--> 159                           % (cmd, exit_status))
    160             elif os.WIFSTOPPED(status):

DistutilsExecError: command 'clang' failed with exit status 1

During handling of the above exception, another exception occurred:

CompileError                              Traceback (most recent call last)
<ipython-input-2-b8ab5105bb50> in <module>()
     10 """
     11 
---> 12 sm = pystan.StanModel(model_code=stan_model_code)

~/anaconda3/lib/python3.7/site-packages/pystan/model.py in __init__(self, file, charset, model_name, model_code, stanc_ret, include_paths, boost_lib, eigen_lib, verbose, obfuscate_model_name, extra_compile_args)
    348 
    349         try:
--> 350             build_extension.run()
    351         finally:
    352             if redirect_stderr:

~/anaconda3/lib/python3.7/distutils/command/build_ext.py in run(self)
    337 
    338         # Now actually compile and link everything.
--> 339         self.build_extensions()
    340 
    341     def check_extensions_list(self, extensions):

~/anaconda3/lib/python3.7/distutils/command/build_ext.py in build_extensions(self)
    446             self._build_extensions_parallel()
    447         else:
--> 448             self._build_extensions_serial()
    449 
    450     def _build_extensions_parallel(self):

~/anaconda3/lib/python3.7/distutils/command/build_ext.py in _build_extensions_serial(self)
    471         for ext in self.extensions:
    472             with self._filter_build_errors(ext):
--> 473                 self.build_extension(ext)
    474 
    475     @contextlib.contextmanager

~/anaconda3/lib/python3.7/distutils/command/build_ext.py in build_extension(self, ext)
    531                                          debug=self.debug,
    532                                          extra_postargs=extra_args,
--> 533                                          depends=ext.depends)
    534 
    535         # XXX outdated variable, kept here in case third-part code

~/anaconda3/lib/python3.7/distutils/ccompiler.py in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends)
    572             except KeyError:
    573                 continue
--> 574             self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
    575 
    576         # Return *all* object filenames, not just the ones we just built.

~/anaconda3/lib/python3.7/distutils/unixccompiler.py in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
    118                        extra_postargs)
    119         except DistutilsExecError as msg:
--> 120             raise CompileError(msg)
    121 
    122     def create_static_lib(self, objects, output_libname,

CompileError: command 'clang' failed with exit status 1

Expected Output:

INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_8b588fdcf67e45da3e1122393e447758 NOW.

PyStan Version:

2.18.0.0. The expected output above was produced with PyStan version 2.17.1.0, which has no problems.

Python Version:

3.7.0. Also fails with 3.6.x.

Here are the dependencies:

cython                    0.28.5
numpy                     1.15.1
pandas                    0.23.4
python                    3.7.0

Operating System:

Above example produced with macOS 10.13.6.

[riddell-stan]

PyStan doesn't yet support Python 3.7. We're working on it.

[justinbois]

I get exactly the same failure with Python 3.6.

[ahartikainen]

Hi, can you try on a new conda environment?

Anaconda might have it's own libc++ / libstdc++ files that are not compatible with osx clang.

[justinbois]

Yes, creating a new conda environment works, both for new environments with Python 3.6.6 and Python 3.7.0, created, respectively with

conda create -n stan python=3.7 pystan=2.18.0.0
conda create -n stan3.6 python=3.6 pystan=2.18.0.0

I had previously gotten that to work. I do think something changed between PyStan 2.17.1.0 and PyStan 2.18.0.0, because with the former, I do not need to create a new environment; it just works with Anaconda. Sorry if I was not clear, but this is the issue I am reporting.

[ahartikainen]

Hi,

Thanks for the info. It really seems a bit bizarre.

In your example you use python=3.6 pystan=2.18.0.0. Is that installing PyStan from conda? (I have not followed conda-forge evolution in the last few days, so I'm not sure if 2.18 is already fixed and published).

2.18 must have a compiler with C++11 support, and osx-clang might have a problem with that. What I mean is that even if vanilla clang version x.yz has support for it macosx clang x.yz is not.

Maybe we can try to find out if this is the problem?

[rgerkin]

I confirm that pystan=2.18.0.0 fails to compile models under a conda environment on macOS, whereas pystan=2.17.1.0 succeeds. The error messages for 2.18.0.0 are different for me than for @justinbois, but possibly related to the same underlying issues, e.g.:

/miniconda3/lib/python3.7/site-packages/pystan/stan/lib/stan_math/lib/boost_1.66.0/boost/smart_ptr/detail/local_counted_base.hpp:110:66: error: 
      no member named 'move' in namespace 'std'

and

/miniconda3/lib/python3.7/site-packages/pystan/stan/lib/stan_math/lib/eigen_3.3.3/Eigen/Core:273:10: fatal error: 'type_traits' file
      not found

[ahartikainen]

This is a compiler problem with C++11.

For the conda environment does conda install gcc work?

There is some incompatibility between libc and libstd.

Could you try on a fresh env?

[rgerkin]

conda install gcc works, and solves the problem for me, allowing model compilation on both pystan 2.17.1.0 and 2.18.0.0 from a conda environment. I do get several hundred compiler warnings, all of the flavor:

warning: section "__datacoal_nt" is deprecated
warning: section "__const_coal" is deprecated
warning: section "__textcoal_nt" is deprecated

But I assume those are generic deprecation warnings that you are already aware of.

This conda problem might be avoidable by including gcc in a environment.yml file that gets used by the conda installations (but not pip ones), although maybe this would need to be modified to only apply to Macs and not Linux/Windows since the latter may already have a reasonable version of gcc and not want to rely on the conda one. For Macs you can probably assume that the version of gcc already on the path is terrible and outdated and not worthy of respect.

[beyondpie]

@justinbois
I tried conda create -n stan python=3.7 pystan=2.18.0.0 under Conda env, and it works for me now.
My MacOS is Mojave with version 10.14.2.

Well, I face this warning when I run the Eight school problem :

INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_19a09b474d1901f191444eaf8a6b8ce2 NOW. /anaconda3/envs/stan_py3-7/lib/python3.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /var/folders/7m/32g80bhx65dg43030t4k9l2w0000gp/T/tmp2g1d05ne/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_3908116714556174198.pyx
But the model is running and finished OK, I think.
Anyone has ideas about the warning, thanks.

[ahartikainen]

It comes from Cython and it is already fixed. The current release is 2.18.1, and it should be already fixed in there.

#525

[yangkky]

I tried

conda create -n stan python=3.7 pystan=2.18.0.0

followed by

conda install gcc

on OS X Mojave 10.14.2, and I'm still getting

CompileError: command 'gcc' failed with exit status 1

[ahartikainen]

Hi, I think there is some bug with default channel on anaconda.

Can you first create environment and activate it, and then, conda install gcc and then pip install pystan.

[yangkky]

Hi, I think there is some bug with default channel on anaconda.

Can you first create environment and activate it, and then, conda install gcc and then pip install pystan.

I did

conda create -n stan python=3.6
conda install gcc
pip install pystan

I still get CompileError: command 'gcc' failed with exit status 1.

[ahartikainen]

That CompileError is coming when you try to install pystan or when you try to compile model?

Have you installed xcode/xcode command line tools + opened xcode to accept all the needed acceptions.

[yangkky]

The CompileError is when I try to compile a model.

I have the xcode command line tools. I don't see xcode so I guess I'll fight with the app store to download that. I had pystan working on this computer before Mojave though?!

[ahartikainen]

Yeah, the command-line-tools should be enough.

Can you uninstall gcc from conda and then try to compile model.

There is some problems with headers/stdlibc++/libc++ etc with Mojave xcode (Mojave moved to libc++ only and some stuff is breaking with this, also I'm not sure about the C++11 thing with default compiler on mojave).

Also Can you search for the error message (if you use Jupyter, error message is in the terminal) what it says. Is it missing ios without conda gcc?

[justinbois]

I ran into this a while ago, and I got this tip from @gchure:

Apple moved where the headers live (i.e. limits.h, etc). If you run into this problem, you can fix it by manually installing them from this package:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

It fixed my problem.

[yangkky]

This is the error message in the terminal:

warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead        
      [-Wstdlibcxx-not-found]       
/var/folders/4x/cwzl6_jx5nvg86y0pxg7nk6r0000gn/T/tmpwj15sb2f/stanfit4linear_c26e92ddbf2fad19476549ac006d52f3_3785948357819386491.cpp:678:10: fatal error:                           
      'ios' file not found          
#include "ios"  

I uninstalled conda gcc, and scipy and seaborn no longer import. The terminal told me to run sudo xcodebuild -license when I tried to compile a stan model. I did that and agreed to the terms, and then it went back to the error above.

So I installed conda gcc again and tried open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg. I still get the 'ios' file not found error.

[ahartikainen]

So I think there are multiple things going on.

So if you can, remove the conda env and create a new one.

Don't install gcc from conda. Make sure gcc --version works and is the default xcode clang compiler.

Run the following open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Install pystan with pip.

[rgerkin]

Going forward, I think support for conda's gcc should be part of testing. More and more people are using Anaconda, especially on non-Linux machines, and many conda packages will install conda's gcc. For that matter, conda's gcc is superior along many dimensions to the native or brew (etc.) gcc on OSX. It is usually just a matter of adding one include or library flag to the gcc call.

[riddell-stan]

@rgerkin I think testing using conda's gcc is a good idea. Do you know of a Python package with a compiled extension module that uses conda's gcc?

[ahartikainen]

See updated toolchain instructions (docs)

Closing due to inactivity. Can be reopened if needed.

[feruzi]

PyStan doesn't yet support Python 3.7. We're working on it.

still pystan don't support python 3.7? i am having the same problem and i am using python 3.7 on ubuntu

[ahartikainen]

PyStan 2.19.1 has python 3.7 wheel for Linux

https://pypi.org/project/pystan/#files

[GittiHab]

I wanted to add my experience after having the same initial problem. It turned out the problem was that I used a conda environment and installed PyStan using pip. So it is crucial that PyStan is installed using conda (e.g. via the conda-forge channel).

conda install -c conda-forge pystan

also installs all dependencies so everything works fine. I am using python 3.7 so that also works.