Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some models (e.g. VAST) fail to compile with newer compilers #297

Open
jkbest2 opened this issue Jun 11, 2019 · 17 comments
Open

Some models (e.g. VAST) fail to compile with newer compilers #297

jkbest2 opened this issue Jun 11, 2019 · 17 comments
Labels
External A problem in an external project that TMB depends on

Comments

@jkbest2
Copy link

jkbest2 commented Jun 11, 2019 •
edited

Issue

I am having trouble getting newer versions of VAST to compile. Others are encountering similar difficulties (e.g. James-Thorson-NOAA/VAST#140). In that issue Jim suggested posting here for more feedback on the compiler errors. I have successfully compiled and fit other models using the same versions of TMB and compilers.

Reproducible example

To reproduce the issue, run

library(TMB)

download.file("https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v8_0_0.cpp", "VAST_v8_0_0.cpp")

compile("VAST_v8_0_0.cpp")

The output from clang++ and g++ available in this Gist, but the three errors from clang++ are

In file included from VAST_v8_0_0.cpp:1:
In file included from /home/jkbest/R/x86_64-pc-linux-gnu-library/3.6/TMB/include/TMB.hpp:53:
In file included from /home/jkbest/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/Eigen/Dense:1:
In file included from /home/jkbest/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/Eigen/Core:371:
/home/jkbest/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/Eigen/src/Core/MathFunctions.h:713:12: error: no matching function for call to 'isfinite'
    return isfinite EIGEN_NOT_A_MACRO (x);
^~~~~~~~

The errors output by g++ are similar, but more verbose.

Version information

OS: Kubuntu Linux 19.04

R: 3.6.0
TMB: 1.7.15
RcppEigen: 0.3.3.5.0

clang++: 8.0.0-3
g++: 8.3.0

I get the same results using clang++ 6 and g++ 6. These are the earliest versions easily available for my distribution.
kaskr added a commit that referenced this issue Jun 12, 2019
@kaskr
Fix isfinite etc #297
03656bb
kaskr added a commit that referenced this issue Jun 12, 2019
@kaskr
Fix isfinite etc #297
9d57c63
@kaskr
Copy link
Owner

kaskr commented Jun 12, 2019

@jkbest2 Should be fixed - please test.

@jkbest2
Copy link
Author

jkbest2 commented Jun 12, 2019 •
edited

Thank you!

VAST now compiles for me using both clang++ 8 and gcc 8. This appears to be resolved.

@jimianelli
Copy link

jimianelli commented Jun 12, 2019 via email

@kaskr
Copy link
Owner

kaskr commented Jun 13, 2019

@jimianelli
This is unlikely a TMB issue. FWIW, the message Symbol not found: ___addtf3* indicates a linkage problem on your Mac system. The library /usr/local/lib/libquadmath.0.dylib seems broken?

@jkbest2
Copy link
Author

jkbest2 commented Jun 13, 2019 •
edited

Edit: Sorry, I see you already installed the correct clang version. Maybe it's a PATH issue and the correct one isn't always being called?

I suspect @kaskr is correct. It looks like compilers on Macs can be a bit finicky, but the required compilers for your OS X and R versions should be available here.

@jimianelli
Copy link

jimianelli commented Jun 13, 2019 via email

@LuisACubillos
Copy link

LuisACubillos commented Jun 13, 2019 via email

@kaskr
Copy link
Owner

kaskr commented Jun 13, 2019

@LuisACubillos
So, you get the VAST compilation error when using the current CRAN version of TMB. This is as expected. Would you mind testing if devtools::install_github("kaskr/adcomp/TMB") solves the problem?

@LuisACubillos
Copy link

LuisACubillos commented Jun 14, 2019 via email

@kaskr
Copy link
Owner

kaskr commented Jun 14, 2019

@LuisACubillos OK thanks for testing - it seems this issue is far from solved.

@jimianelli
Copy link

jimianelli commented Jun 14, 2019 via email

@grantdadams
Copy link

grantdadams commented Jun 29, 2019 via email

@sgaichas
Copy link

sgaichas commented Aug 6, 2019

If someone has a better solution for macOS, please provide an update.

I had the same problem described above on macOS, but used this workaround to run VAST; hope this helps. TMB installed fine with clang6, and examples worked both from CRAN and using devtools::install_github("kaskr/adcomp/TMB"), but VAST compilation failed for v 8.2. (I was able to get VAST v 4 to work as Jim described above.)

However, I changed my ~/.R/Makevars after installing TMB and VAST to the following, and now VAST v 8.2 can run the simple example here.

##CC=/usr/local/clang6/bin/clang
##CXX=/usr/local/clang6/bin/clang++
CC=gcc
CXX=/usr/local/bin/g++-9
CXX1X=/usr/local/clang6/bin/clang++
CXX98=/usr/local/clang6/bin/clang++
CXX11=/usr/local/clang6/bin/clang++
CXX14=/usr/local/clang6/bin/clang++
CXX17=/usr/local/clang6/bin/clang++
LDFLAGS=-L/usr/local/clang6/lib

My configuration:

NECLWHSGAICHAS-2: sgaichas$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] FishStatsUtils_2.2.0 mapdata_2.3.0 maps_3.3.0 VAST_3.1.0
[5] TMB_1.7.15

loaded via a namespace (and not attached):
[1] RANN_2.6.1 lattice_0.20-38 TMBhelper_1.1.0 grid_3.5.2
[5] ThorsonUtilities_1.0 INLA_18.07.12 sp_1.3-1 Matrix_1.2-16
[9] splines_3.5.2 tools_3.5.2 splancs_2.01-40 compiler_3.5.2
[13] PBSmapping_2.72.1

@sgaichas sgaichas mentioned this issue Aug 6, 2019
Open
@gavinfay
Copy link

Can confirm that Sarah's solution here for installing TMB and successfully compiling VAST 8 also worked for me using R 3.6.1 and clang7. (thanks Sarah!)

@seanhardison1 seanhardison1 mentioned this issue Feb 10, 2020
Closed
@JenM1 JenM1 mentioned this issue Feb 10, 2020
Closed
@yukio-takeuchi
Copy link

I have a similar trouble when compiling VAST_8_5_0.cpp from the R command

TMB::compile("VAST_v8_5_0.cpp")

On my Mac (MacOS 10.15.3) with Apple clang++

$ clang++ -v
Apple clang version 11.0.0 (clang-1100.0.33.17)
Target: x86_64-apple-darwin19.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

and R-3.6.2 installed by source by homebrew.

On the other hand , when I used Sarah's approach to specify

CXX=/usr/local/bin/g++-9

in ~/.R/Makevars to use g++ compiler installed from source by homebrew,

TMB::compile("VAST_v8_5_0.cpp")

finished without error.

More generally I found the 'TMB::compile("VAST_v***.cpp")' failed for
VAST_v5_0_0.cpp and newer ones if Apple's clang++ compiler is used.

Particular differences between VAST_v5_0_0.cpp and newer ones and previous VAST_v*.cpp is
the use of "std::complex"

vector< std::complex<Type> > eigenvalues_B_ff = B_ff.eigenvalues();

In fact compilation errors, when clang++ is used to compile VAST_v8_5_0.cpp, comes from

  1. Error related to Implicit cast from int to double or float eg.
VAST_v8_5_0.cpp:688:18: error: call to 'sqrt' is ambiguous
    Range_raw1 = sqrt(8) / exp( logkappa1 );   // Range = approx. distance @ 10% correlation

This error is very common for clang(++)

  1. Errors probably related to instantiation of std::complex<>
    Eg.
In file included from VAST_v8_5_0.cpp:1:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/TMB.hpp:53:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/Eigen/Dense:1:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/Eigen/Core:80:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/complex:677:24: error: no matching function for call to 'fmax'
    _Tp __logbw = logb(fmax(fabs(__c), fabs(__d)));
                       ^~~~
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/EigenSolver.h:542:74: note: in instantiation of function template specialization
      'std::__1::operator/<CppAD::AD<double> >' requested here
        ComplexScalar cc = ComplexScalar(Scalar(0),-m_matT.coeff(n-1,n)) / ComplexScalar(m_matT.coeff(n-1,n-1)-p,q);
                                                                         ^
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/EigenSolver.h:447:7: note: in instantiation of member function
      'Eigen::EigenSolver<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >::doComputeEigenvectors' requested here
      doComputeEigenvectors();
      ^
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/EigenSolver.h:156:7: note: in instantiation of function template specialization
      'Eigen::EigenSolver<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >::compute<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >' requested here
      compute(matrix.derived(), computeEigenvectors);
      ^
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/MatrixBaseEigenvalues.h:39:12: note: in instantiation of function template specialization
      'Eigen::EigenSolver<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >::EigenSolver<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >' requested here
    return EigenSolver<PlainObject>(m_eval, false).eigenvalues();
           ^
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/MatrixBaseEigenvalues.h:70:81: note: in instantiation of member function
      'Eigen::internal::eigenvalues_selector<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1>, false>::run' requested here
  return internal::eigenvalues_selector<Derived, NumTraits<Scalar>::IsComplex>::run(derived());
                                                                                ^
VAST_v8_5_0.cpp:439:58: note: in instantiation of member function 'Eigen::MatrixBase<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >::eigenvalues' requested here
    vector< std::complex<Type> > eigenvalues_B_ff = B_ff.eigenvalues();
                                                         ^
VAST_v8_5_0.cpp:727:10: note: in instantiation of function template specialization 'calculate_B<CppAD::AD<double> >' requested here
  B_ff = calculate_B( VamConfig(0), n_f1, VamConfig(1), Chi_fr, Psi_fr, jnll_comp(13) );
         ^
/usr/local/lib/R/3.6/site-library/TMB/include/tmb_core.hpp:1135:6: note: in instantiation of member function 'objective_function<CppAD::AD<double> >::operator()'
      requested here
    F(); // Run through user template (modifies reportvector)
     ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/math.h:532:15: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'double' for 1st argument
extern double fmax(double, double);
              ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1217:46: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'float' for 1st argument
inline _LIBCPP_INLINE_VISIBILITY float       fmax(float __lcpp_x, float __lcpp_y) _NOEXCEPT             {return ::fmaxf(__lcpp_x, __lcpp_y);}
                                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1218:46: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'long double' for 1st argument
inline _LIBCPP_INLINE_VISIBILITY long double fmax(long double __lcpp_x, long double __lcpp_y) _NOEXCEPT {return ::fmaxl(__lcpp_x, __lcpp_y);}
                                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1228:1: note: candidate template ignored: substitution
      failure [with _A1 = CppAD::AD<double>, _A2 = CppAD::AD<double>]: no type named 'type' in 'std::__1::__lazy_enable_if<false,
      std::__1::__promote<CppAD::AD<double>, CppAD::AD<double>, void> >'
fmax(_A1 __lcpp_x, _A2 __lcpp_y) _NOEXCEPT
^

First one is easy to solve by changing
sqrt(8) to sqrt(8.0) or by adding a trivial new function

double sqrt(int x){
  return sqrt(double(x));
}

As for second one, probably similar problem was reported here and posted [their fix] (su2code/SU2#351) for their library.
In essence, they claimed that "complex" header from llvm uses some function specifically assumes their arguments to be double. They caused error when std::complexCppAD::AD was instantiated.

I confirmed, similar error related to std:comlex happens, when I try compiling more simple artificial code

// Simple linear regression.
#include <TMB.hpp>
template<class Type>
Type objective_function<Type>::operator() ()
{
  DATA_VECTOR(Y);
  DATA_VECTOR(x);
  PARAMETER(a);
  PARAMETER(b);
  PARAMETER(logSigma);
  std::complex<Type> c1(3.0, 4.0);
  std::complex<Type> c2(1.0, 1.56);
  std::complex<Type> c3=c1 / c2;
  ADREPORT(exp(2*logSigma));
  Type nll = -sum(dnorm(Y, a+b*x, exp(logSigma), true));
  Rcout << "nll : " << std::endl;
  return nll;
}

Similar error was produced and its compilation was failed

In file included from test.cpp:2:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/TMB.hpp:53:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/Eigen/Dense:1:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/Eigen/Core:80:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/complex:677:24: error: no matching function for call to 'fmax'
    _Tp __logbw = logb(fmax(fabs(__c), fabs(__d)));
                       ^~~~
test.cpp:13:28: note: in instantiation of function template specialization 'std::__1::operator/<CppAD::AD<double> >' requested here
  std::complex<Type> c3=c1 / c2;
                           ^
/usr/local/lib/R/3.6/site-library/TMB/include/tmb_core.hpp:1135:6: note: in instantiation of member function 'objective_function<CppAD::AD<double> >::operator()'
      requested here
    F(); // Run through user template (modifies reportvector)
     ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/math.h:532:15: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'double' for 1st argument
extern double fmax(double, double);
              ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1217:46: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'float' for 1st argument
inline _LIBCPP_INLINE_VISIBILITY float       fmax(float __lcpp_x, float __lcpp_y) _NOEXCEPT             {return ::fmaxf(__lcpp_x, __lcpp_y);}
                                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1218:46: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'long double' for 1st argument
inline _LIBCPP_INLINE_VISIBILITY long double fmax(long double __lcpp_x, long double __lcpp_y) _NOEXCEPT {return ::fmaxl(__lcpp_x, __lcpp_y);}
                                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1228:1: note: candidate template ignored: substitution
      failure [with _A1 = CppAD::AD<double>, _A2 = CppAD::AD<double>]: no type named 'type' in 'std::__1::__lazy_enable_if<false,
      std::__1::__promote<CppAD::AD<double>, CppAD::AD<double>, void> >'
fmax(_A1 __lcpp_x, _A2 __lcpp_y) _NOEXCEPT

In addition if I put a line
CXX=g++-9
in ~/.R/Makevars
TMB::compile command finished without any error.

@yukio-takeuchi yukio-takeuchi mentioned this issue Feb 14, 2020
Closed
@kaskr
Copy link
Owner

kaskr commented Feb 14, 2020

@yukio-takeuchi
The problem is not limited to Mac.
I can replicate the issue on a Linux system using any version of clang up to 11.0.0. (However, the LLVM version of the C++ standard library -stdlib has to be explicitely selected):

TMB:::compile("VAST_v8_0_0.cpp", CXX="clang++ -stdlib=libc++ -std=c++98")

To 'fix' the compilation I switch to the gcc standard library implementation -std=libstdc++. The following works:

TMB:::compile("VAST_v8_0_0.cpp", CXX="clang++ -stdlib=libstdc++ -std=c++98")

Is this an option on Mac as well?

@yukio-takeuchi
Copy link

yukio-takeuchi commented Feb 15, 2020 via email

@sgaichas sgaichas mentioned this issue Mar 17, 2020
Closed
@euclaise euclaise added the External A problem in an external project that TMB depends on label Jul 31, 2020
@jlmorano jlmorano mentioned this issue Aug 19, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
External A problem in an external project that TMB depends on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@LuisACubillos @jimianelli @yukio-takeuchi @kaskr @jkbest2 @gavinfay @sgaichas @grantdadams @euclaise