Install Fails on Compilation Error

[Sonictherocketman]

Hello. I have a Docker build that installs this and several other packages that has been building successfully until the last day or two. Currently I get this error when installing:

#11 393.3 make: *** [Makefile:1379: lme4.ts] Error 1
#11 393.3 make: Target 'all' not remade because of errors.
#11 393.3 * installing *source* package ‘lme4’ ...
#11 393.3 ** package ‘lme4’ successfully unpacked and MD5 sums checked
#11 393.3 ** using staged installation
#11 393.3 ** libs
#11 393.3 using C++ compiler: ‘g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0’
#11 393.3 make[1]: Entering directory '/tmp/RtmpXRjB9x/R.INSTALL16261ecc92f9/lme4/src'
#11 393.3 g++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG  -I'/usr/local/lib/R/site-library/Rcpp/include' -I'/usr/local/lib/R/site-library/RcppEigen/include' -I/usr/local/include   -DNDEBUG -DEIGEN_DONT_VECTORIZE -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c external.cpp -o external.o
#11 393.3 In file included from predModule.h:13,
#11 393.3                  from external.cpp:8:
#11 393.3 lme4CholmodDecomposition.h:20:72: error: expected template-name before ‘<’ token
#11 393.3    20 |     class lme4CholmodDecomposition : public Eigen::CholmodDecomposition<_MatrixType, _UpLo> {
#11 393.3       |                                                                        ^
#11 393.3 lme4CholmodDecomposition.h:20:72: error: expected ‘{’ before ‘<’ token
#11 393.3 In file included from predModule.h:13,
#11 393.3                  from external.cpp:8:
#11 393.3 lme4CholmodDecomposition.h: In function ‘SEXPREC* lme4::Eigen_cholmod_wrap(const lme4::lme4CholmodDecomposition<Eigen::SparseMatrix<Scalar> >&)’:
#11 393.3 lme4CholmodDecomposition.h:77:15: error: ‘cholmod_factor’ does not name a type
#11 393.3    77 |         const cholmod_factor* f = obj.factor();
#11 393.3       |               ^~~~~~~~~~~~~~
#11 393.3 lme4CholmodDecomposition.h:78:13: error: ‘f’ was not declared in this scope
#11 393.3    78 |         if (f->minor < f->n)
#11 393.3       |             ^
#11 393.3 lme4CholmodDecomposition.h:82:36: error: ‘f’ was not declared in this scope
#11 393.3    82 |         ::Rcpp::S4 ans(std::string(f->is_super ? "dCHMsuper" : "dCHMsimpl"));
#11 393.3       |                                    ^
#11 393.3 In file included from external.cpp:8:
#11 393.3 predModule.h: At global scope:
#11 393.3 predModule.h:40:23: error: field ‘d_L’ has incomplete type ‘lme4::merPredD::ChmDecomp’ {aka ‘lme4::lme4CholmodDecomposition<Eigen::SparseMatrix<double, 0, int> >’}
#11 393.3    40 |         ChmDecomp     d_L;
#11 393.3       |                       ^~~
#11 393.3 In file included from predModule.h:13,
#11 393.3                  from external.cpp:8:
#11 393.3 lme4CholmodDecomposition.h:20:11: note: declaration of ‘lme4::merPredD::ChmDecomp’ {aka ‘class lme4::lme4CholmodDecomposition<Eigen::SparseMatrix<double, 0, int> >’}
#11 393.3    20 |     class lme4CholmodDecomposition : public Eigen::CholmodDecomposition<_MatrixType, _UpLo> {
#11 393.3       |           ^~~~~~~~~~~~~~~~~~~~~~~~
#11 393.3 external.cpp: In function ‘SEXPREC* glmerAGQ(SEXP, SEXP, SEXP, SEXP, SEXP, SEXP, SEXP)’:
#11 393.3 external.cpp:437:18: error: invalid use of incomplete type ‘const ChmDecomp’ {aka ‘const class lme4::lme4CholmodDecomposition<Eigen::SparseMatrix<double, 0, int> >’}
#11 393.3   437 |         if (pp->L().factor()->nzmax !=  q)
#11 393.3       |             ~~~~~^~
#11 393.3 In file included from predModule.h:13,
#11 393.3                  from external.cpp:8:
#11 393.3 lme4CholmodDecomposition.h:20:11: note: declaration of ‘lme4::merPredD::ChmDecomp’ {aka ‘class lme4::lme4CholmodDecomposition<Eigen::SparseMatrix<double, 0, int> >’}
#11 393.3    20 |     class lme4CholmodDecomposition : public Eigen::CholmodDecomposition<_MatrixType, _UpLo> {
#11 393.3       |           ^~~~~~~~~~~~~~~~~~~~~~~~
#11 393.3 external.cpp:439:49: error: invalid use of incomplete type ‘const ChmDecomp’ {aka ‘const class lme4::lme4CholmodDecomposition<Eigen::SparseMatrix<double, 0, int> >’}
#11 393.3   439 |         const Ar1         sd(MAr1((double*)pp->L().factor()->x, q).inverse());
#11 393.3       |                                            ~~~~~^~
#11 393.3 In file included from predModule.h:13,
#11 393.3                  from external.cpp:8:
#11 393.3 lme4CholmodDecomposition.h:20:11: note: declaration of ‘lme4::merPredD::ChmDecomp’ {aka ‘class lme4::lme4CholmodDecomposition<Eigen::SparseMatrix<double, 0, int> >’}
#11 393.3    20 |     class lme4CholmodDecomposition : public Eigen::CholmodDecomposition<_MatrixType, _UpLo> {
#11 393.3       |           ^~~~~~~~~~~~~~~~~~~~~~~~
#11 393.3 make[1]: *** [/usr/local/lib/R/etc/Makeconf:200: external.o] Error 1
#11 393.3 make[1]: Leaving directory '/tmp/RtmpXRjB9x/R.INSTALL16261ecc92f9/lme4/src'
#11 393.3 ERROR: compilation failed for package ‘lme4’
#11 393.3 * removing ‘/usr/local/lib/R/site-library/lme4’
#11 393.3 cat: pbkrtest.out: No such file or directory
#11 393.3 cat: car.out: No such file or directory
#11 393.3 cat: rstatix.out: No such file or directory
#11 393.3 cat: FactoMineR.out: No such file or directory
#11 393.3 cat: ggpubr.out: No such file or directory
#11 393.3 cat: factoextra.out: No such file or directory
#11 393.5 
#11 393.5 The downloaded source packages are in
#11 393.5 	‘/tmp/downloaded_packages’
#11 393.5 Error: installation of 7 packages failed:
#11 393.5   ‘lme4’, ‘pbkrtest’, ‘car’, ‘rstatix’, ‘FactoMineR’, ‘ggpubr’, ‘factoextra’
#11 393.5 In addition: Warning message:
#11 393.5 In install.packages(pkgs, ...) : installation of 7 packages failed:
#11 393.5   ‘lme4’, ‘pbkrtest’, ‘car’, ‘rstatix’, ‘FactoMineR’, ‘ggpubr’, ‘factoextra’
------

It seems to be an issue with lme4 but I'm not sure. Any help is appreciated. Nothing else has changed in the project.

[Sonictherocketman]

Here is the relevant section of the Dockerfile

FROM rocker/rstudio:latest

RUN apt-get clean && apt-get update
RUN apt-get install -y \
    default-jdk \
    libpoppler-cpp-dev \
    libtesseract-dev \
    libleptonica-dev \
    libglpk-dev \
    libgsl-dev \
    python3

WORKDIR /app
COPY packages.csv .
COPY install-packages.sh .
RUN ./install-packages.sh
# ^ this part installs lme4 and other R-libs using install2.r

[bbolker]

I'm not sure what install2.r is. I see that install2.r is from littler ... You should be aware that new versions of RcppEigen (0.3.3.9.4) and lme4 (1.1-35 and 1.1-35.1) have recently been released, and that the previous version of lme4 (1.1.34) is not forward compatible with the new version of RcppEigen; my guess is that you are somehow getting mismatched versions. (The most likely case would be that you have new RcppEigen and old lme4; if it were the other way around you should be getting an "insufficient version" error rather than a compilation error ...)

Hope that helps.

[SHogenboom]

@Sonictherocketman ; have you been able to resolve your issue? Even with the latest fix mentioned by @saraemoore I still encounter the same issue. Would really like to know how to fix it :)

[bbolker]

As I suspected, I can replicate this error via:

remotes::install_version("RcppEigen", "0.3.3.9.4")
remotes::install_version("lme4", "1.1-34")

Whereas installing the newer version of lme4 in conjunction with RcppEigen 0.3.3.9.4 (remotes::install_version("lme4", "1.1-35.1")) works fine (so should version 1.1-35; 1.1-35.1 was a patch fix for unrelated issues that came out a couple of days later).

I don't know what mirrors you're using/where you're retrieving the source tarball from; r-project.org definitely has the updated source tarball . I do see that r2u has the newer version of RcppEigen.

[SHogenboom]

@bbolker ; thanks for your reply! The issue arrose when trying to install a packages which relies on lme4 as a dependency:

- R -e "remotes::install_gitlab('r-packages/rosetta@dev’)"

I’m not familiar with the CI/CD intricacies myself so not sure how to resolve the version issues when dealing with external dependencies.

[bbolker]

My first try would be to see if there is a package cache you can clear so everything gets reinstalled. My second would be to force installation of RcppEigen and lme4, in that order (install.packages("RcppEigen", type = "source"); install.packages("lme4", type = "source")), although that should be overkill (installing an up-to-date version of lme4 should force updating of RcppEigen ...)

[bbolker]

I also suspect that this issue will resolve itself in a few days as the new versions propagate to wherever, FWIW (but I understand that you want to fix this now rather than crossing your fingers and waiting)

[SHogenboom]

Installing lme4 prior to the rosetta package resulted in a new error… An issue with: installation of package ‘nloptr’ had non-zero exit status.

For now we can get around with a manual publish. So I’ll indeed wait a couple of days to see if the issue resolves itself.
Thanks very much for thinking along!

[bbolker]

I'm probably not going to dig into this further, but I would welcome any further comments/information about the source of the problem (i.e., where is the old version of lme4 being retrieved from, and is there a way to force installation from a newer source?)

[saraemoore]

The solution that worked for me within a GitHub Action was to add upgrade: 'TRUE' to the r-lib/actions/setup-r-dependencies@v2 step, i.e.:

      - uses: r-lib/actions/setup-r-dependencies@v2
        with:
          upgrade: 'TRUE'

Since this worked to force installation of lme4 1.1-35.1 instead of 1.1-34 -- and since both installations were from source according to the action's logs (i.e. 1.1-34 wasn't preferred because it was available precompiled, and both versions were apparently available from whatever repository the action was pulling from -- RSPM?) -- I'm assuming that whatever global/shared cache is used by r-lib/actions/setup-r-dependencies@v2 was the culprit in my particular case. Note also that, in this case, the GitHub Action failed the first time it was run, so this was not a cache specific to my GitHub repo.

[bbolker]

(So much for not looking into it further.) tl;dr The root of the problem seems to be that r2u has the updated version of RcppEigen but lme4 hasn't been updated yet. For GitHub Actions (but not for the original Docker-related question??) adding lme4 to extra-packages (however one does that??) should resolve the problem, I think. For Docker my guess would be that you should add an explicit Rscript -e 'install.packages("lme4", type = "source")' ...

This is probably helpful: https://github.com/r-lib/actions/tree/v2/setup-r-dependencies#installing-the-latest-dependencies

Typically, if there is a binary build of an older package version available, and that satisfies all version requirements, then it will be preferred over a later source R package.

If your package does need a later version of a dependency, then you need to explicitly require a newer version in the DESCRIPTION file. Alternatively, if you only want to run the CI jobs with a later version, without a formal version requirement, then add the package to the extra-packages parameter. setup-r-dependencies installs the latest versions of these packages.

My guess/assumption is that the actions are pulling from r2u, which at present still has "old" lme4 and "new" RcppEigen ...

$ apt-cache policy r-cran-lme4
r-cran-lme4:
  Installed: 1.1-34-1.ca2204.1
  Candidate: 1.1-34-1.ca2204.1
  Version table:
 *** 1.1-34-1.ca2204.1 700
        700 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 Packages
        100 /var/lib/dpkg/status
     1.1-28-1 500
        500 http://apt.pop-os.org/ubuntu jammy/universe amd64 Packages


$  apt-cache policy r-cran-rcppeigen
r-cran-rcppeigen:
  Installed: 0.3.3.9.3-1.ca2204.1
  Candidate: 0.3.3.9.4-1.ca2204.1
  Version table:
     0.3.3.9.4-1.ca2204.1 700
        700 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 Packages
 *** 0.3.3.9.3-1.ca2204.1 100
        100 /var/lib/dpkg/status
     0.3.3.9.1-1 500
        500 http://apt.pop-os.org/ubuntu jammy/universe amd64 Packages

PS I'm still a little puzzled because the error messages look like they're coming from a source install of lme4 1.1-34, in which case my explanation doesn't make sense ... ??

[Sonictherocketman]

For what it's worth my original issue seems to be solved by "waiting it out". Thanks for all the movement on this. I wasn't aware that the R-lang packaging environment was structured this way.

[bbolker]

I'm going to go ahead and close this, people can re-open if it seems to need further discussion. (One final note is that I think it was probably RSPM, which I wasn't even really aware of, that was actually serving most of the out-of-sync binaries that were causing the problems ...