New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rstan compilation error under Boost 1.64 #441

Closed
eddelbuettel opened this Issue Aug 12, 2017 · 44 comments

Comments

Projects
None yet
@eddelbuettel

eddelbuettel commented Aug 12, 2017

Summary:

rstan does not build under Boost 1.64

Description:

I try to update the BH package once in a while. We are currently at 1.62, Boost upstream is at 1.64.
With a candidate release (currently in a branch of my bh repo, happy to provide a .tar.gz for your testing) we fail to compile rstan.

Reproducible Steps:

Just build rstan with BH upgraded to 1.64 (which includes three gentle mods to upstream, none of which should matter here as they do not include Boost Phoenix)

Current Output:

In file included from /usr/local/lib/R/site-library/BH/include/boost/phoenix/core/argument.hpp:14:0,
                 from /usr/local/lib/R/site-library/BH/include/boost/phoenix/core.hpp:15,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/include/phoenix_core.hpp:11,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/support/make_component.hpp:15,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/support/meta_compiler.hpp:20,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/qi/meta_compiler.hpp:14,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/qi/action/action.hpp:14,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/qi/action.hpp:14,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/qi.hpp:14,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/include/qi.hpp:16,
                 from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/lang/grammars/expression07_grammar.hpp:4,
                 from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/lang/grammars/expression_grammar.hpp:5,
                 from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/lang/grammars/expression_grammar_def.hpp:4,
                 from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/lang/grammars/expression_grammar_inst.cpp:1,
                 from lang__grammars__expression_grammar_inst.cpp:18:
/usr/local/lib/R/site-library/BH/include/boost/phoenix/core/expression.hpp:63:21: note:   initializing argument 4 of ‘static boost::phoenix::expr_ext<Actor, Tag, A>::type boost::phoenix::expr_ext<Actor, Tag, A>::make(typename boost::call_traits<A>::param_type ...) [with Actor = boost::phoenix::actor; Tag = boost::phoenix::detail::tag::function_eval; A = {stan::lang::binary_op_expr, boost::phoenix::actor<boost::spirit::attribute<0> >, boost::phoenix::actor<boost::spirit::argument<0> >, char*, char*, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::reference_wrapper<std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> > > >, 0l> >}; boost::phoenix::expr_ext<Actor, Tag, A>::type = boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::phoenix::detail::tag::function_eval, boost::proto::argsns_::list6<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<stan::lang::binary_op_expr>, 0l>, boost::phoenix::actor<boost::spirit::attribute<0> >, boost::phoenix::actor<boost::spirit::argument<0> >, boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<char*>, 0l>, boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<char*>, 0l>, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::reference_wrapper<std::__cxx11::basic_stringstream<char> > >, 0l> > >, 6l> >]’
         static type make(typename call_traits<A>::param_type... a)
                     ^~~~
In file included from /usr/local/lib/R/site-library/BH/include/boost/phoenix/function/detail/cpp03/preprocessed/function_operator.hpp:12:0,
                 from /usr/local/lib/R/site-library/BH/include/boost/phoenix/function/detail/cpp03/function_operator.hpp:6,
                 from /usr/local/lib/R/site-library/BH/include/boost/phoenix/function/function.hpp:58,
                 from /usr/local/lib/R/site-library/BH/include/boost/phoenix/function.hpp:11,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/include/phoenix_function.hpp:11,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/support/terminal.hpp:18,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/support/common_terminals.hpp:15,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/qi/auto/auto.hpp:13,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/qi/auto.hpp:15,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/home/qi.hpp:15,
                 from /usr/local/lib/R/site-library/BH/include/boost/spirit/include/qi.hpp:16,
                 from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/lang/grammars/expression07_grammar.hpp:4,
                 from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/lang/grammars/expression_grammar.hpp:5,
                 from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/lang/grammars/expression_grammar_def.hpp:4,
                 from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/lang/grammars/expression_grammar_inst.cpp:1,
                 from lang__grammars__expression_grammar_inst.cpp:18:
/usr/local/lib/R/site-library/BH/include/boost/phoenix/function/detail/cpp03/preprocessed/function_operator_10.hpp:91:86: error: invalid conversion from ‘const char*’ to ‘param_type {aka char*}’ [-fpermissive]
             return detail::expression::function_eval<F, A0 , A1 , A2 , A3 , A4>::make(f, a0 , a1 , a2 , a3 , a4);
                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~

and lots more like it.

Expected Output:

Standard build,

RStan Version:

Current CRAN version: 2.16.2

R Version:

Current version 3.4.1

Operating System:

Ubuntu Linux 16.10

@bob-carpenter

This comment has been minimized.

Show comment
Hide comment
@bob-carpenter

bob-carpenter Aug 12, 2017

Contributor

Thanks. We'll get that fixed ASAP. The Boost Spirit Qi parser depends heavily on the sequence abstraction (fusion) and on the lazy evaluation modules (phoenix), so this is where things usually break on upgrades.

Contributor

bob-carpenter commented Aug 12, 2017

Thanks. We'll get that fixed ASAP. The Boost Spirit Qi parser depends heavily on the sequence abstraction (fusion) and on the lazy evaluation modules (phoenix), so this is where things usually break on upgrades.

@bob-carpenter

This comment has been minimized.

Show comment
Hide comment
@bob-carpenter

bob-carpenter Aug 12, 2017

Contributor

@bgoodri --- Mitzi or I can do this over the weekend and hopefully we'll be able to come up with something that works with the current Boost and the new one.

Contributor

bob-carpenter commented Aug 12, 2017

@bgoodri --- Mitzi or I can do this over the weekend and hopefully we'll be able to come up with something that works with the current Boost and the new one.

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 12, 2017

Super, appreciate the quick reply.

This is not too urgent. I had one much more severe failure (Boost ublas now needs an additonal header related to serilization, so I had ~ 15 packages fails; fixed with a one-line change in the BH version of Boost 1.64). Now that that is out of the way ...

All it takes, I guess, is for us to be in sync -- ie for me to wait til you're ready so that you can update your package(s) once a new BH is up. Sounds like a plan?

eddelbuettel commented Aug 12, 2017

Super, appreciate the quick reply.

This is not too urgent. I had one much more severe failure (Boost ublas now needs an additonal header related to serilization, so I had ~ 15 packages fails; fixed with a one-line change in the BH version of Boost 1.64). Now that that is out of the way ...

All it takes, I guess, is for us to be in sync -- ie for me to wait til you're ready so that you can update your package(s) once a new BH is up. Sounds like a plan?

@bob-carpenter

This comment has been minimized.

Show comment
Hide comment
@bob-carpenter

bob-carpenter Aug 12, 2017

Contributor

Yes, that'll work. We'll try to get a version that works with both the current Boost and the new Boost. We've usually been able to do that before.

Contributor

bob-carpenter commented Aug 12, 2017

Yes, that'll work. We'll try to get a version that works with both the current Boost and the new Boost. We've usually been able to do that before.

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 12, 2017

I can probably fold my 1.64 branch into my master, or just provide a tar.gz for you.

eddelbuettel commented Aug 12, 2017

I can probably fold my 1.64 branch into my master, or just provide a tar.gz for you.

@bob-carpenter

This comment has been minimized.

Show comment
Hide comment
@bob-carpenter

bob-carpenter Aug 12, 2017

Contributor

Let me see if I can fix this on the Stan side rather than the RStan side. Usually the problem is there with a new version of Boost.

Contributor

bob-carpenter commented Aug 12, 2017

Let me see if I can fix this on the Stan side rather than the RStan side. Usually the problem is there with a new version of Boost.

@bgoodri

This comment has been minimized.

Show comment
Hide comment
@bgoodri

bgoodri Aug 12, 2017

Contributor
Contributor

bgoodri commented Aug 12, 2017

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 12, 2017

FWIW BH 1.64.0-0 (a release candidate) is now the master branch giving you access to what I used. It has three innocent changes (two based on R constraints + customs; one a Boost ublas bug).

I have one other minor issue with another package that also needs sorting out, so no rush. If we all get to this next week we'd be doing great. It is August after all ;-)

eddelbuettel commented Aug 12, 2017

FWIW BH 1.64.0-0 (a release candidate) is now the master branch giving you access to what I used. It has three innocent changes (two based on R constraints + customs; one a Boost ublas bug).

I have one other minor issue with another package that also needs sorting out, so no rush. If we all get to this next week we'd be doing great. It is August after all ;-)

@bgoodri

This comment has been minimized.

Show comment
Hide comment
@bgoodri

bgoodri Aug 12, 2017

Contributor
Contributor

bgoodri commented Aug 12, 2017

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 14, 2017

As an FYI, the other issue that came up was a simple lack of RNG seeding which has been addressed.

So I could ship BH to CRAN, but I have to no problem waiting a few days for you as I presume this is progressing at your side?

eddelbuettel commented Aug 14, 2017

As an FYI, the other issue that came up was a simple lack of RNG seeding which has been addressed.

So I could ship BH to CRAN, but I have to no problem waiting a few days for you as I presume this is progressing at your side?

@bgoodri

This comment has been minimized.

Show comment
Hide comment
@bgoodri

bgoodri Aug 14, 2017

Contributor
Contributor

bgoodri commented Aug 14, 2017

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 20, 2017

How is it looking? Did the extra day(s) help? Any visibility regarding a future upload?

eddelbuettel commented Aug 20, 2017

How is it looking? Did the extra day(s) help? Any visibility regarding a future upload?

@bgoodri

This comment has been minimized.

Show comment
Hide comment
@bgoodri

bgoodri Aug 21, 2017

Contributor

I think it is going to be okay, but they haven't quite released the new Stan yet. There was a lot of consternation over a change to the normal RNG and the original build error we have only been able to fix by adding another define in Makevars.

Contributor

bgoodri commented Aug 21, 2017

I think it is going to be okay, but they haven't quite released the new Stan yet. There was a lot of consternation over a change to the normal RNG and the original build error we have only been able to fix by adding another define in Makevars.

@bob-carpenter

This comment has been minimized.

Show comment
Hide comment
@bob-carpenter

bob-carpenter Aug 21, 2017

Contributor
Contributor

bob-carpenter commented Aug 21, 2017

@bgoodri

This comment has been minimized.

Show comment
Hide comment
@bgoodri

bgoodri Aug 21, 2017

Contributor
Contributor

bgoodri commented Aug 21, 2017

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 21, 2017

Thanks for the update, much appreciated. If it seems that a release is "imminent", I'd be happy to wait. BH releases are not super urgent. That said, I'd also like to get it out of the way :) So keep me posted.

eddelbuettel commented Aug 21, 2017

Thanks for the update, much appreciated. If it seems that a release is "imminent", I'd be happy to wait. BH releases are not super urgent. That said, I'd also like to get it out of the way :) So keep me posted.

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 23, 2017

How are we doing? I have in the meantime

  • updated BH to Boost 1.65.0,
  • ran another set of tests,
  • contacted another maintainer another required (but tiny) change
  • that change has been made

So whenever you're ready ...

eddelbuettel commented Aug 23, 2017

How are we doing? I have in the meantime

  • updated BH to Boost 1.65.0,
  • ran another set of tests,
  • contacted another maintainer another required (but tiny) change
  • that change has been made

So whenever you're ready ...

@bob-carpenter

This comment has been minimized.

Show comment
Hide comment
@bob-carpenter

bob-carpenter Aug 24, 2017

Contributor

@eddelbuettel

Thanks for being so patient. I'm not exactly sure where the process is at on our side, but this one was complicated in that it created a bunch of breaking changes. I think Ben's just decided to put in the compiler flag to fix the issue in RStan since we haven't come up with a general fix.

Contributor

bob-carpenter commented Aug 24, 2017

@eddelbuettel

Thanks for being so patient. I'm not exactly sure where the process is at on our side, but this one was complicated in that it created a bunch of breaking changes. I think Ben's just decided to put in the compiler flag to fix the issue in RStan since we haven't come up with a general fix.

@bgoodri

This comment has been minimized.

Show comment
Hide comment
@bgoodri

bgoodri Aug 24, 2017

Contributor
Contributor

bgoodri commented Aug 24, 2017

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Aug 24, 2017

@bob-carpenter No worries. Some packages are complicated and need time.

@bgoodri So shall I go ahead, get BH 1.65.0-1 onto CRAN and you will then follow with Stan and rstan?

eddelbuettel commented Aug 24, 2017

@bob-carpenter No worries. Some packages are complicated and need time.

@bgoodri So shall I go ahead, get BH 1.65.0-1 onto CRAN and you will then follow with Stan and rstan?

@bgoodri

This comment has been minimized.

Show comment
Hide comment
@bgoodri

bgoodri Aug 24, 2017

Contributor
Contributor

bgoodri commented Aug 24, 2017

@kleinschmidt

This comment has been minimized.

Show comment
Hide comment
@kleinschmidt

kleinschmidt Sep 21, 2017

I just ran into this issue today. Adding that compiler flag to my Makevars worked.

kleinschmidt commented Sep 21, 2017

I just ran into this issue today. Adding that compiler flag to my Makevars worked.

@EliaKunz

This comment has been minimized.

Show comment
Hide comment
@EliaKunz

EliaKunz Sep 22, 2017

having the same issue. trying to run the rstan installation on ubuntu linux. is this going to be fixed in rstan 2.17?

EliaKunz commented Sep 22, 2017

having the same issue. trying to run the rstan installation on ubuntu linux. is this going to be fixed in rstan 2.17?

@bgoodri

This comment has been minimized.

Show comment
Hide comment
@bgoodri

bgoodri Sep 22, 2017

Contributor
Contributor

bgoodri commented Sep 22, 2017

@EliaKunz

This comment has been minimized.

Show comment
Hide comment
@EliaKunz

EliaKunz Sep 22, 2017

Awesome! Thanks

EliaKunz commented Sep 22, 2017

Awesome! Thanks

@odino

This comment has been minimized.

Show comment
Hide comment
@odino

odino Oct 9, 2017

Hello! Is v2.17 published on CRAN? Cant seem to find it anywhere :)

odino commented Oct 9, 2017

Hello! Is v2.17 published on CRAN? Cant seem to find it anywhere :)

@kuperov

This comment has been minimized.

Show comment
Hide comment
@kuperov

kuperov Oct 18, 2017

Hi, a quick FYI for anyone else who might be blocked by this (Linux users who can't get binary builds). One workaround is to manually uninstall BH (remove.packages('BH')) then install an older BH version, available here. Then you should be able to install the rstan package normally.

kuperov commented Oct 18, 2017

Hi, a quick FYI for anyone else who might be blocked by this (Linux users who can't get binary builds). One workaround is to manually uninstall BH (remove.packages('BH')) then install an older BH version, available here. Then you should be able to install the rstan package normally.

@odino

This comment has been minimized.

Show comment
Hide comment
@odino

odino Oct 19, 2017

BTW can confirm an older version of BH works, this is how we have it running:

https://github.com/namshi/docker-r-base/blob/master/Dockerfile#L6-L7

odino commented Oct 19, 2017

BTW can confirm an older version of BH works, this is how we have it running:

https://github.com/namshi/docker-r-base/blob/master/Dockerfile#L6-L7

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Oct 19, 2017

(Off-topic here: you probably want one or at most two RUN statements in your Dockerfile; install.packages() takes a vector. Rocker files themselves use install.r pkg1 pkg2 .... See the various Dockerfile examples in the Rocker project.)

eddelbuettel commented Oct 19, 2017

(Off-topic here: you probably want one or at most two RUN statements in your Dockerfile; install.packages() takes a vector. Rocker files themselves use install.r pkg1 pkg2 .... See the various Dockerfile examples in the Rocker project.)

@pgensler

This comment has been minimized.

Show comment
Hide comment
@pgensler

pgensler Oct 22, 2017

@odino hmm, this is not working for my docker file, I get a g++ compile error:
dockerfile is attached, open to feedback on it.

Docker_filev1.txt

g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
make: *** [lang__grammars__statement_grammar_inst.o] Error 4
/usr/local/lib/R/etc/Makeconf:168: recipe for target 'lang__grammars__statement_grammar_inst.o' failed
ERROR: compilation failed for package ‘rstan’
* removing ‘/usr/local/lib/R/site-library/rstan’
ERROR: dependency ‘rstan’ is not available for package ‘prophet’
* removing ‘/usr/local/lib/R/site-library/prophet’

The downloaded source packages are in
        ‘/tmp/RtmpMX3ny9/downloaded_packages’
Warning messages:
1: In install.packages("prophet") :
  installation of package ‘rstan’ had non-zero exit status
2: In install.packages("prophet") :
  installation of package ‘prophet’ had non-zero exit status

pgensler commented Oct 22, 2017

@odino hmm, this is not working for my docker file, I get a g++ compile error:
dockerfile is attached, open to feedback on it.

Docker_filev1.txt

g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
make: *** [lang__grammars__statement_grammar_inst.o] Error 4
/usr/local/lib/R/etc/Makeconf:168: recipe for target 'lang__grammars__statement_grammar_inst.o' failed
ERROR: compilation failed for package ‘rstan’
* removing ‘/usr/local/lib/R/site-library/rstan’
ERROR: dependency ‘rstan’ is not available for package ‘prophet’
* removing ‘/usr/local/lib/R/site-library/prophet’

The downloaded source packages are in
        ‘/tmp/RtmpMX3ny9/downloaded_packages’
Warning messages:
1: In install.packages("prophet") :
  installation of package ‘rstan’ had non-zero exit status
2: In install.packages("prophet") :
  installation of package ‘prophet’ had non-zero exit status
@odino

This comment has been minimized.

Show comment
Hide comment
@odino

odino Oct 22, 2017

odino commented Oct 22, 2017

@pgensler

This comment has been minimized.

Show comment
Hide comment
@pgensler

pgensler Oct 22, 2017

@odino odd, on FROM rocker/r-base:latest

I get:

installing to /usr/local/lib/R/site-library/htmltools/libs
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (htmltools)
* installing *source* package ‘xml2’ ...
** package ‘xml2’ successfully unpacked and MD5 sums checked
Package libxml-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libxml-2.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libxml-2.0' found
Package libxml-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libxml-2.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libxml-2.0' found
Using PKG_CFLAGS=
Using PKG_LIBS=-lxml2
ERROR: configuration failed for package ‘xml2’
* removing ‘/usr/local/lib/R/site-library/xml2’
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because libxml-2.0 was not found. Try installing:
 * deb: libxml2-dev (Debian, Ubuntu, etc)
 * rpm: libxml2-devel (Fedora, CentOS, RHEL)
 * csw: libxml2_dev (Solaris)
If libxml-2.0 is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a libxml-2.0.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
Error in install.packages(pkgs = f, lib = lib, repos = if (isMatchingFile(f)) NULL else rep,  :
  installation of package ‘xml2’ had non-zero exit status

FYI, I am trying to run a container with RStudio, if you want me to run the rocker/rstudio image for debugging, I can do that.

pgensler commented Oct 22, 2017

@odino odd, on FROM rocker/r-base:latest

I get:

installing to /usr/local/lib/R/site-library/htmltools/libs
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (htmltools)
* installing *source* package ‘xml2’ ...
** package ‘xml2’ successfully unpacked and MD5 sums checked
Package libxml-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libxml-2.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libxml-2.0' found
Package libxml-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libxml-2.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libxml-2.0' found
Using PKG_CFLAGS=
Using PKG_LIBS=-lxml2
ERROR: configuration failed for package ‘xml2’
* removing ‘/usr/local/lib/R/site-library/xml2’
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because libxml-2.0 was not found. Try installing:
 * deb: libxml2-dev (Debian, Ubuntu, etc)
 * rpm: libxml2-devel (Fedora, CentOS, RHEL)
 * csw: libxml2_dev (Solaris)
If libxml-2.0 is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a libxml-2.0.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
Error in install.packages(pkgs = f, lib = lib, repos = if (isMatchingFile(f)) NULL else rep,  :
  installation of package ‘xml2’ had non-zero exit status

FYI, I am trying to run a container with RStudio, if you want me to run the rocker/rstudio image for debugging, I can do that.

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Oct 22, 2017

In the container, do apt-get install libxml2-dev just as it suggests. Then try again.

eddelbuettel commented Oct 22, 2017

In the container, do apt-get install libxml2-dev just as it suggests. Then try again.

@pgensler

This comment has been minimized.

Show comment
Hide comment
@pgensler

pgensler Oct 22, 2017

Not good:

virtual memory exhausted: Cannot allocate memory
make: *** [lang__grammars__statement_grammar_inst.o] Error 1
/usr/lib/R/etc/Makeconf:168: recipe for target 'lang__grammars__statement_grammar_inst.o' failed
ERROR: compilation failed for package ‘rstan’
* removing ‘/usr/local/lib/R/site-library/rstan’
ERROR: dependency ‘rstan’ is not available for package ‘prophet’
* removing ‘/usr/local/lib/R/site-library/prophet’

The downloaded source packages are in
        ‘/tmp/RtmppASSXo/downloaded_packages’
Warning messages:
1: In install.packages("prophet") :
  installation of package ‘rstan’ had non-zero exit status
2: In install.packages("prophet") :
  installation of package ‘prophet’ had non-zero exit status
 ---> 8872b6d06cc1
Removing intermediate container e69aa2ca9ed5
Successfully built 8872b6d06cc1
Successfully tagged my_image:latest

@eddelbuettel Dirk, I'm not entirely sure how to fix this, but I'm assuming this is due to all my Rscript's creating layers and not cleaning them up properly?

pgensler commented Oct 22, 2017

Not good:

virtual memory exhausted: Cannot allocate memory
make: *** [lang__grammars__statement_grammar_inst.o] Error 1
/usr/lib/R/etc/Makeconf:168: recipe for target 'lang__grammars__statement_grammar_inst.o' failed
ERROR: compilation failed for package ‘rstan’
* removing ‘/usr/local/lib/R/site-library/rstan’
ERROR: dependency ‘rstan’ is not available for package ‘prophet’
* removing ‘/usr/local/lib/R/site-library/prophet’

The downloaded source packages are in
        ‘/tmp/RtmppASSXo/downloaded_packages’
Warning messages:
1: In install.packages("prophet") :
  installation of package ‘rstan’ had non-zero exit status
2: In install.packages("prophet") :
  installation of package ‘prophet’ had non-zero exit status
 ---> 8872b6d06cc1
Removing intermediate container e69aa2ca9ed5
Successfully built 8872b6d06cc1
Successfully tagged my_image:latest

@eddelbuettel Dirk, I'm not entirely sure how to fix this, but I'm assuming this is due to all my Rscript's creating layers and not cleaning them up properly?

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Oct 22, 2017

virtual memory exhausted: Cannot allocate memory

If you are on something like AWS or another minuscule machine, up the RAM. We are talking C++ here, and complicated at that. You want "normal" amounts of RAM. 4gb should do.

eddelbuettel commented Oct 22, 2017

virtual memory exhausted: Cannot allocate memory

If you are on something like AWS or another minuscule machine, up the RAM. We are talking C++ here, and complicated at that. You want "normal" amounts of RAM. 4gb should do.

@pgensler

This comment has been minimized.

Show comment
Hide comment
@pgensler

pgensler Oct 22, 2017

I have 16gb on my laptop, so I would assume that should be more than enough. Is there some bash flag I need to be setting in my script to force allocate more memory, or clean up temp files after the packages have installed to free up container space?

pgensler commented Oct 22, 2017

I have 16gb on my laptop, so I would assume that should be more than enough. Is there some bash flag I need to be setting in my script to force allocate more memory, or clean up temp files after the packages have installed to free up container space?

@pgensler

This comment has been minimized.

Show comment
Hide comment
@pgensler

pgensler Oct 22, 2017

@eddelbuettel @odino OK so allocating 8GB to docker instead of 2GB made the container build from rocker/r-base:latest, which is good, but I'd still like to use RStudio with this container preferably with the rocker/tidyverse image if possible.

Not to bloat this thread, but even after using FROM rocker/tidyverse:latest, and using the following code:

RUN Rscript -e 'remove.packages("BH")'
RUN Rscript -e 'install.packages("https://cran.r-project.org/src/contrib/Archive/BH/BH_1.62.0-1.tar.gz")'
RUN Rscript -e 'install.packages("rstan")'
RUN Rscript -e 'install.packages("prophet")'

-docker will install the correct package BH_1.62.0-1.tar.gz, but when attempting to install rstan, it tries to install rstan from https://cran.rstudio.com/src/contrib/rstan_2.16.2.tar.gz, which seems to throw the error from before:

g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
/usr/local/lib/R/etc/Makeconf:168: recipe for target 'lang__grammars__statement_grammar_inst.o' failed
make: *** [lang__grammars__statement_grammar_inst.o] Error 4
ERROR: compilation failed for package ‘rstan’
* removing ‘/usr/local/lib/R/site-library/rstan’

The downloaded source packages are in
        ‘/tmp/RtmpP7pmDm/downloaded_packages’
Warning message:
In install.packages("rstan") :
  installation of package ‘rstan’ had non-zero exit status

So is there any way to get the FROM rocker/tidyverse:latest to work with this? I understand the images are different, and thus can cause some headache when building, but still, it would be nice to have a painfree process to build a container. I have attached the part of the log so you can see what is exactly happening in the container build process as described above.
dockerfile_output.txt

pgensler commented Oct 22, 2017

@eddelbuettel @odino OK so allocating 8GB to docker instead of 2GB made the container build from rocker/r-base:latest, which is good, but I'd still like to use RStudio with this container preferably with the rocker/tidyverse image if possible.

Not to bloat this thread, but even after using FROM rocker/tidyverse:latest, and using the following code:

RUN Rscript -e 'remove.packages("BH")'
RUN Rscript -e 'install.packages("https://cran.r-project.org/src/contrib/Archive/BH/BH_1.62.0-1.tar.gz")'
RUN Rscript -e 'install.packages("rstan")'
RUN Rscript -e 'install.packages("prophet")'

-docker will install the correct package BH_1.62.0-1.tar.gz, but when attempting to install rstan, it tries to install rstan from https://cran.rstudio.com/src/contrib/rstan_2.16.2.tar.gz, which seems to throw the error from before:

g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
/usr/local/lib/R/etc/Makeconf:168: recipe for target 'lang__grammars__statement_grammar_inst.o' failed
make: *** [lang__grammars__statement_grammar_inst.o] Error 4
ERROR: compilation failed for package ‘rstan’
* removing ‘/usr/local/lib/R/site-library/rstan’

The downloaded source packages are in
        ‘/tmp/RtmpP7pmDm/downloaded_packages’
Warning message:
In install.packages("rstan") :
  installation of package ‘rstan’ had non-zero exit status

So is there any way to get the FROM rocker/tidyverse:latest to work with this? I understand the images are different, and thus can cause some headache when building, but still, it would be nice to have a painfree process to build a container. I have attached the part of the log so you can see what is exactly happening in the container build process as described above.
dockerfile_output.txt

@dmenne

This comment has been minimized.

Show comment
Hide comment
@dmenne

dmenne Oct 22, 2017

I got a similar installation working;

https://github.com/dmenne/gastro-docker/blob/master/Dockerfile

Don't complain about the seemingly chaotic sequence of apt-get and install-R. This was a lot of experimenting until I had the dependencies right. 4 GB was barely enough to build it locally, on the free Docker Hub/Cloud I cannot build this one.

dmenne commented Oct 22, 2017

I got a similar installation working;

https://github.com/dmenne/gastro-docker/blob/master/Dockerfile

Don't complain about the seemingly chaotic sequence of apt-get and install-R. This was a lot of experimenting until I had the dependencies right. 4 GB was barely enough to build it locally, on the free Docker Hub/Cloud I cannot build this one.

@pgensler

This comment has been minimized.

Show comment
Hide comment
@pgensler

pgensler Oct 22, 2017

@dmenne just as an FYI--this dockerfile should build stan in the tidyverse (at least as of 6 months ago), and I am unable to build it, even commennting out the
# COPY R/Makevars /root/.R/Makevars
--https://hub.docker.com/r/andrewheiss/tidyverse-rstanarm/~/dockerfile/. I'm definitely out of ideas on how to fix/troubleshoot my dockerfile, so if anyone is willing to help tinker it to make it work......see attached.
Dockerfile_merged.txt

pgensler commented Oct 22, 2017

@dmenne just as an FYI--this dockerfile should build stan in the tidyverse (at least as of 6 months ago), and I am unable to build it, even commennting out the
# COPY R/Makevars /root/.R/Makevars
--https://hub.docker.com/r/andrewheiss/tidyverse-rstanarm/~/dockerfile/. I'm definitely out of ideas on how to fix/troubleshoot my dockerfile, so if anyone is willing to help tinker it to make it work......see attached.
Dockerfile_merged.txt

@feuerbach

This comment has been minimized.

Show comment
Hide comment
@feuerbach

feuerbach Nov 21, 2017

I just ran into this. @kuperov's suggestion worked for me. These are the commands I used:

remove.packages('BH')
devtools::install_url("https://cran.r-project.org/src/contrib/Archive/BH/BH_1.62.0-1.tar.gz")
install.packages('rstan')

feuerbach commented Nov 21, 2017

I just ran into this. @kuperov's suggestion worked for me. These are the commands I used:

remove.packages('BH')
devtools::install_url("https://cran.r-project.org/src/contrib/Archive/BH/BH_1.62.0-1.tar.gz")
install.packages('rstan')
@ashander

This comment has been minimized.

Show comment
Hide comment
@ashander

ashander Dec 15, 2017

🙌 @feuerbach + @kuperov

Also for other's info: I thought based on earlier comments from @kleinschmidt and @bgoodri that the following would work

 $ cat .R/Makevars 
CXXFLAGS = -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION

(but did not for me with BH 1.65 )

ashander commented Dec 15, 2017

🙌 @feuerbach + @kuperov

Also for other's info: I thought based on earlier comments from @kleinschmidt and @bgoodri that the following would work

 $ cat .R/Makevars 
CXXFLAGS = -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION

(but did not for me with BH 1.65 )

@jabranham

This comment has been minimized.

Show comment
Hide comment
@jabranham

jabranham Jan 2, 2018

rstan 2.17.2 is on CRAN and I don't get this error anymore. This issue can probably be closed @bob-carpenter.

Thanks for the update!

jabranham commented Jan 2, 2018

rstan 2.17.2 is on CRAN and I don't get this error anymore. This issue can probably be closed @bob-carpenter.

Thanks for the update!

@eddelbuettel

This comment has been minimized.

Show comment
Hide comment
@eddelbuettel

eddelbuettel Jan 2, 2018

Yep.

And for what it is worth I also blogged about how to avoid a lot a Docker errors if you just simplify your life with binaries. The post has a working example and demo. The Dockerfile is in essence just (some comments remove, see blog for full copy)

FROM rocker/r-apt:xenial
MAINTAINER "Carl Boettiger and Dirk Eddelbuettel" rocker-maintainers@eddelbuettel.com
RUN apt-get update && apt-get install -y --no-install-recommends r-cran-rstan
CMD ["R"]

and can be used as-is (once saved as rocker/rstan:local)

$ docker run --rm -ti rocker/rstan:local Rscript -e 'library(rstan)'
Loading required package: ggplot2
Loading required package: StanHeaders
rstan (Version 2.16.2, packaged: 2017-07-03 09:24:58 UTC, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
rstan_options(auto_write = TRUE)
options(mc.cores = parallel::detectCores())
$

eddelbuettel commented Jan 2, 2018

Yep.

And for what it is worth I also blogged about how to avoid a lot a Docker errors if you just simplify your life with binaries. The post has a working example and demo. The Dockerfile is in essence just (some comments remove, see blog for full copy)

FROM rocker/r-apt:xenial
MAINTAINER "Carl Boettiger and Dirk Eddelbuettel" rocker-maintainers@eddelbuettel.com
RUN apt-get update && apt-get install -y --no-install-recommends r-cran-rstan
CMD ["R"]

and can be used as-is (once saved as rocker/rstan:local)

$ docker run --rm -ti rocker/rstan:local Rscript -e 'library(rstan)'
Loading required package: ggplot2
Loading required package: StanHeaders
rstan (Version 2.16.2, packaged: 2017-07-03 09:24:58 UTC, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
rstan_options(auto_write = TRUE)
options(mc.cores = parallel::detectCores())
$
@jabranham

This comment has been minimized.

Show comment
Hide comment
@jabranham

jabranham Jan 2, 2018

Thanks, I saw that. Very interesting stuff! The other post you had (about ccache) has saved me tons of time already.

Unfortunately, I don't use a .deb based distro :-(

jabranham commented Jan 2, 2018

Thanks, I saw that. Very interesting stuff! The other post you had (about ccache) has saved me tons of time already.

Unfortunately, I don't use a .deb based distro :-(

@bgoodri bgoodri closed this Jan 2, 2018

ariddell pushed a commit to ariddell/httpstan that referenced this issue Jan 7, 2018

ariddell added a commit to stan-dev/httpstan that referenced this issue Jan 7, 2018

rgiordan pushed a commit to rgiordan/rstan that referenced this issue Apr 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment