Visual C++ 14.0 breaks on vector addition

[bob-carpenter]

From https://github.com/belteshassar on stan-dev/stan#2110:

Summary:

Vector addition gives a compiler error in Microsoft Visual C++ 14.0. This does not reproduce when using g++ on Ubuntu on Windows.

Description:

I use pystan with Python 3.5. I have managed to compile and run a variety of models both on Windows using Visual C++ 14.0 and on the new Windows subsystem for Linux (WSL). When building a more complex model with vector parameters I managed to get a compiler error in Visual C++

error C2672: 'add': no matching overloaded function found

I narrowed the issue down to vector addition. I can reproduce the issue using a vector addition in either of the transformed data, transformed parameters, or model blocks.

I don't know if this can be reproduced on older versions of Python using older compilers. I use Visual C++ 14.0 since it's the recommended compiler for building Python extensions for Python 3.5 on Windows.

Reproducible Steps:

Here is a simplified model that can reproduce the issue:

data {
  int<lower=0> I;
  simplex[I] y;
}
transformed data {
  vector[I] ones;
  for (i in 1:I) {
    ones[i] = 1.0;
  }
}
parameters {
  vector[I] alpha;
}
model {
  y ~ dirichlet(alpha + ones);
}

Current Output:

error C2672: 'add': no matching overloaded function found

Additional Information:

OS: Windows 10 Pro 64-bit Build 14393.447
Visual C++ 14.0
Python 3.5.2
pystan 2.12.0.0

Current Version:

v2.12.0

[belteshassar]

I'd use g++, as suggested by @bob-carpenter on the original issue, but that's unsupported by Python so it may or may not work. I understand if this is not a priority issue, though.

[syclik]

@belteshassar, do you know how to instantiate the compiler with all the options from the command line? If so, mind posting here?

[belteshassar]

@syclik Not sure what you mean. Cython handles everything with the compiler.

Anyway, I decided to run pystan on Windows Subsystem for Linux, which works great.. I've managed to compile and run medium sized models with no issues. Also, python multiprocessing works fine on WSL so I can use the full power of my machine.

[syclik]

@belteshassar, sorry -- missed the comment.

I don't know what Cython does to call the MSVC compiler. If you can extract that info, I can try to run that command on a Windows box and see if I can make progress. I don't know enough about Cython to figure that out easily.

[ahartikainen]

Hi, due to current compiler situation with python3, pystan does not work on windows with python 3.5/3.6. Currently, I have no idea how to fix this. There are some notes on accumulator.

Errors are probably there due to some bug in MSVC.

Errors are:

error C2672: 'stan::math::accumulator<double>::add': no matching overloaded function found


error C2784: 'void stan::math::accumulator<double>::add(const std::vector<S,std::allocator<_Other>> &)': could not deduce template argument for 'const std::vector<S,std::allocator<_Other>> &' from 'stan::math::var'

error C2784: 'void stan::math::accumulator<double>::add(const Eigen::Matrix<S,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &)': could not deduce template argument for 'const Eigen::Matrix<S,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &' from 'stan::math::var'

error C2893: Failed to specialize function template 'boost::disable_if<boost::is_arithmetic<S>,boost::enable_if<boost::is_same<S,T>,void>::type>::type stan::math::accumulator<T>::add(const S &)'

error C2893: Failed to specialize function template 'boost::enable_if<boost::is_arithmetic<S>,void>::type stan::math::accumulator<double>::add(S)'

https://github.com/stan-dev/math/blob/develop/stan/math/prim/mat/fun/accumulator.hpp

The reason for the accumulator error is probably on these lines with add(m(i)); or add(xs[i]);

  /**
   * Add each entry in the specified matrix, vector, or row vector
   * of values to the buffer.
   *
   * @tparam S type of values in matrix
   * @tparam R number of rows in matrix
   * @tparam C number of columns in matrix
   * @param m Matrix of values to add
   */
  template <typename S, int R, int C>
  void add(const Eigen::Matrix<S, R, C>& m) {
    for (int i = 0; i < m.size(); ++i)
      add(m(i));
  }

or

  /**
   * Recursively add each entry in the specified standard vector
   * to the buffer.  This will allow vectors of primitives,
   * auto-diff variables to be added; if the vector entries
   * are collections, their elements are recursively added.
   *
   * @tparam S Type of value to recursively add.
   * @param xs Vector of entries to add
   */
  template <typename S>
  void add(const std::vector<S>& xs) {
    for (size_t i = 0; i < xs.size(); ++i)
      add(xs[i]);
  }

[rmax]

I think this is because Eigen compatibility with MSVC. I replaced vector and matrix types with real [] and it works fine.

[ahartikainen]

Hi,

I did some testing with MSVC 14 (2015) / 14.1 (2017). I could not get this working yet. This is a compiler bug, but I tried to surpass that.

It looks like precompiler on MSVC can not deduce the correct type from functions and custom types. Does it only check one step backwards? The real reason is probably something more complicated.

On Eigen site there is an article:

Writing Functions Taking Eigen Types as Parameters

It says in the page that

Eigen's use of expression templates results in potentially every expression being of a different type.

This could or could not be the one of the many problems that MSVC has with our templated functions.

---

---

With the following code I tested some combinations and here are the results.

MSVC 14.1 (2017)
Python 3.6
PyStan 15.0.1 (pystan/model.py: from distutils.core import Extension --> from setuptools import Extension with updated setuptools)

model_code = """
data {
  int<lower=0> M;
  int<lower=0> N;
  vector[N] y;
  matrix[N, M] A;
  matrix[N, N] Sigma;
}
transformed data {
}
parameters {
  vector[N] y2;
  vector[M] x;
}
model {
  y ~ multi_normal(y2 + A * x, Sigma);
}"""

I created a pystan.stanc instance and edited the line 245 in template function log_prob

lp_accum__.add(multi_normal_log<propto__>(y, add(y2,multiply(A,x)), Sigma));

id	Compiled	Code
1	🔴 False	lp_accum__.add(multi_normal_log<propto__>(y, add(y2,multiply(A,x)), Sigma));
2	True	//lp_accum__.add(multi_normal_log<propto__>(y, add(y2,multiply(A,x)), Sigma));
3	True	multiply(A,x);
4	🔴 False	add(y2,multiply(A,x));
5	🔴 False	add(y2,y);
6	True	for (int i = 0; i < y2.size(); ++i) y2(i) + y(i);
7	🔴 False	multi_normal_log<propto__>(y, y2, Sigma);
8	True	normal_log<propto__>(y, y2, 1);
9	🔴 False	normal_log<propto__>(y, multiply(A,x), 1);
10	True	lp_accum__.add(1);
11	True	lp_accum__.add(normal_log<propto__>(y, y2, 1));

---

Here are some common errors.

The add function can be found here edit. Maybe I forgot the link to stan-math (?).

1. Original (3 errors)

lp_accum__.add(multi_normal_log<propto__>(y, add(y2,multiply(A,x)), Sigma));` :

1.1

c:\users\ahartikainen\appdata\local\temp\tmp4t0b5t44\stanc_approach_520a921b93884c162bb6e86e901b28db.hpp(246): error C2784: 'Eigen::Matrix<boost::math::tools::promote_args<RT1,RT2,float,float,float,float>::type,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> stan::math::add(const T1 &,const Eigen::Matrix<T2,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &)': could not deduce template argument for 'const Eigen::Matrix<T2,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &' from 'Eigen::Matrix<stan::math::var,-1,-1,0,-1,-1>'

extracted:

could not deduce template argument for 
'const Eigen::Matrix<T2,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &' 
from
'Eigen::Matrix<stan::math::var,-1,-1,0,-1,-1>'

1.2.

c:\users\ahartikainen\appdata\local\temp\tmp4t0b5t44\stanc_approach_520a921b93884c162bb6e86e901b28db.hpp(246): error C2782: 'Eigen::Matrix<boost::math::tools::promote_args<RT1,RT2,float,float,float,float>::type,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> stan::math::add(const Eigen::Matrix<T1,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &,const T2 &)': template parameter '_Cols' is ambiguous

extracted:

template parameter '_Cols' is ambiguous

1.3

c:\users\ahartikainen\appdata\local\temp\tmp4t0b5t44\stanc_approach_520a921b93884c162bb6e86e901b28db.hpp(246): error C2784: 'Eigen::Matrix<boost::math::tools::promote_args<RT1,RT2,float,float,float,float>::type,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> stan::math::add(const Eigen::Matrix<T1,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &,const T2 &)': could not deduce template argument for 'const Eigen::Matrix<T1,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &' from 'Eigen::Matrix<Eigen::NumTraits<stan::math::var>::Real,-1,1,0,-1,1>'

extracted:

could not deduce template argument for 
'const Eigen::Matrix<T1,R,C,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &' 
from 
'Eigen::Matrix<Eigen::NumTraits<stan::math::var>::Real,-1,1,0,-1,1>'

2 Multinormal

multi_normal_log<propto__>(y, y2, Sigma)

multi_normal_lpdf.hpp(109): error C2672: 'trace_inv_quad_form_ldlt': no matching overloaded function found
C:\Users\ahartikainen\Miniconda3\envs\stan36\lib\site-packages\pystan\stan\lib\stan_math_2.15.0\stan/math/prim/mat/prob/multi_normal_lpdf.hpp(109): error C2782: 'boost::enable_if_c<stan::is_var<T>::value||stan::is_var<T3>::value,stan::math::var>::type stan::math::trace_inv_quad_form_ldlt(const stan::math::LDLT_factor<T,R,C> &,const Eigen::Matrix<T2,R2,C2,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &)': template parameter '_Cols' is ambiguous
C:\Users\ahartikainen\Miniconda3\envs\stan36\lib\site-packages\pystan\stan\lib\stan_math_2.15.0\stan/math/rev/mat/fun/trace_inv_quad_form_ldlt.hpp(177): note: see declaration of 'stan::math::trace_inv_quad_form_ldlt'
C:\Users\ahartikainen\Miniconda3\envs\stan36\lib\site-packages\pystan\stan\lib\stan_math_2.15.0\stan/math/prim/mat/prob/multi_normal_lpdf.hpp(109): error C2784: 'boost::enable_if_c<stan::is_var<T>::value||stan::is_var<T3>::value,stan::math::var>::type stan::math::trace_inv_quad_form_ldlt(const stan::math::LDLT_factor<T,R,C> &,const Eigen::Matrix<T2,R2,C2,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &)': could not deduce template argument for 'const Eigen::Matrix<T2,R2,C2,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &' from 'Eigen::Matrix<Eigen::NumTraits<stan::math::var>::Real,-1,1,0,-1,1>'
C:\Users\ahartikainen\Miniconda3\envs\stan36\lib\site-packages\pystan\stan\lib\stan_math_2.15.0\stan/math/rev/mat/fun/trace_inv_quad_form_ldlt.hpp(177): note: see declaration of 'stan::math::trace_inv_quad_form_ldlt'
C:\Users\ahartikainen\Miniconda3\envs\stan36\lib\site-packages\pystan\stan\lib\stan_math_2.15.0\stan/math/prim/mat/prob/multi_normal_lpdf.hpp(109): error C2784: 'boost::enable_if_c<!stan::is_var<T>::value&&!stan::is_var<T2>::value,boost::math::tools::promote_args<RT1,RT2,float,float,float,float>::type>::type stan::math::trace_inv_quad_form_ldlt(const stan::math::LDLT_factor<T1,R2,C2> &,const Eigen::Matrix<T2,R2,C2,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &)': could not deduce template argument for 'const Eigen::Matrix<T2,R2,C2,|_Rows==&&_Cols!=?:_Cols==&&_Rows!=?:,_Rows,_Cols> &' from 'Eigen::Matrix<Eigen::NumTraits<stan::math::var>::Real,-1,1,0,-1,1>'
C:\Users\ahartikainen\Miniconda3\envs\stan36\lib\site-packages\pystan\stan\lib\stan_math_2.15.0\stan/math/prim/mat/fun/trace_inv_quad_form_ldlt.hpp(27): note: see declaration of 'stan::math::trace_inv_quad_form_ldlt'

3. Normal

normal_log<propto__>(y, multiply(A,x), 1);

3.1

C:\Users\ahartikainen\Miniconda3\envs\stan36\lib\site-packages\pystan\stan\lib\stan_math_2.15.0\stan/math/prim/scal/prob/normal_lpdf.hpp(91): error C2440: 'initializing': cannot convert from 'Eigen::Matrix<double,-1,-1,0,-1,-1>' to 'T_partials_return'

Extracted

C2440: cannot convert from
'Eigen::Matrix<double,-1,-1,0,-1,-1>'
to
'T_partials_return'

[bob-carpenter]

@ahartikainen I couldn't follow the last comment. What is the table supposed to be and what are teh numbered parts after it?

Before we throw a lot of resources down this rabbit hole, is there a reason we need to compile with MSVC?

[bob-carpenter]

I also believe Eigen itself is having problems with MSVC compatibility.

[seantalts]

@ariddell As far as you know, should PyStan work on Windows with Python3? I think @ahartikainen is having an issue here where he is not able to get it to work and I can't tell if it's because he's using a non-standard installation?

[ariddell]

PyStan works fine on Windows with Python 3.5 and higher. We have CI testing (appveyor) of all commits.

[ariddell]

I can't imagine why something would work in cmdstan on Windows but not on PyStan on Windows. If this is the case it's a bug against PyStan.

One way to debug this would be to create a PR with a test which should fail because of the bug and have appveyor test it. I find this easier than installing windows in a VM.

[ahartikainen]

@bob-carpenter, the table shows if I can compile C++ code generated with pystan.stanc and then run with pystan.StanModel(stanc_ret=...). I have changed only the line 245.
This table shows that it is the line 245 that causes the compilation error on Windows. I did test with different components (matrix, vector) and some will work (example multiply(A, x) which uses Eigen::BaseMatrix<derived> (edit.) and some don't (add(y, multiply(A,x)) which uses Eigen::Matrix<T_ , C, R>.

The numbered parts after the table shows the common error messages for codes seen on the table. I have copied "only relevant" information. I know it's not a good idea to cut the log and if it's needed I can copy the full log here.

On Windows, currently Pystan is only supported Python 3.5>. Python 3.5 & 3.6 can only use MSVC 14 / 14.1 to compile C/C++. Due to UCRT copyright problem ( https://mingwpy.github.io/issues.html#the-vs-14-2015-runtime ) we don't have any open source compiler (mingw) to compile external modules.

PyStan itself work fine on Windows, only that some vector and matrix calculations do not compile. One can use arrays.

@seantalts I have installed PyStan on conda environment with pip / conda.

I'm not sure how much we should put the effort on this problem, because it is due to MSVC bug (in 14>).

[bob-carpenter]

Thanks---just what I was asking. Sounds like more motivation to produce a standalone Stan server based on a more robust C++ toolkit (like the Rtools distribution of g++ for Windows). But even that C++ toolkit is terrible compared to running on Mac OS or Linux---it's prone to blow out memory during linking, crash during compilation with segfaults on any but a single optimization level, etc. It's been no end of hassle getting things to work in that environment. Yet I think most of our users are on Windows.