Build Error

[binxiangni]

I try to build and check this package locally in RStudio and an error comes out: Error: package or namespace load failed for ‘RcppArmadillo’ in dyn.load(file, DLLpath = DLLpath, ...):. Another warning message is "WARNING: support for OpenMP requires C++11/C++14; add -std=c++11 or -std=c++14 to compiler flags". I google it and there is someone meeting the same problem during installation, but I don't know where to fix yet.

When I check files, there are two files R/RcppExports.R and src/RcppExports.cpp possibly modified by RStudio Build and a new file inst/include/RcppArmadilloLapack.h. I have no idea whether they are to do with the error. Could anyone do me a favor?

[eddelbuettel]

Which package? RcppArmadillo? Can bisect to a working commit?

Do you have access to another machine besides macOS?

[kevinushey]

What is the entire error message that you're seeing? Are you seeing, for example, something like:

** testing if installed package can be loaded
Error: package or namespace load failed for ‘RcppArmadillo’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/kevin/Library/R/3.4/library/RcppArmadillo/libs/RcppArmadillo.so':
  dlopen(/Users/kevin/Library/R/3.4/library/RcppArmadillo/libs/RcppArmadillo.so, 6): Symbol not found: _RcppArmadillo_fastLm
  Referenced from: /Users/kevin/Library/R/3.4/library/RcppArmadillo/libs/RcppArmadillo.so
  Expected in: flat namespace
 in /Users/kevin/Library/R/3.4/library/RcppArmadillo/libs/RcppArmadillo.so
Error: loading failed

I think this is because RcppArmadillo has an entry that confuses the new version of Rcpp::compileAttributes(), which RStudio calls when building the package:

fastLmPure <- function(X, y) {

    stopifnot(is.matrix(X), is.numeric(y), nrow(y)==nrow(X))

    .Call( "RcppArmadillo_fastLm", X, y, PACKAGE = "RcppArmadillo" )
}

The R native routine skeleton sees that, and through that Rcpp::compileAttributes() ends up emitting into RcppExports.cpp:

RcppExport SEXP RcppArmadillo_fastLm(SEXP, SEXP);

static const R_CallMethodDef CallEntries[] = {
    {"_RcppArmadillo_fastLm", (DL_FUNC) &_RcppArmadillo_fastLm, 2},
    {"_RcppArmadillo_armadillo_version", (DL_FUNC) &_RcppArmadillo_armadillo_version, 1},
    {"_RcppArmadillo_armadillo_set_seed_random", (DL_FUNC) &_RcppArmadillo_armadillo_set_seed_random, 0},
    {"_RcppArmadillo_armadillo_set_seed", (DL_FUNC) &_RcppArmadillo_armadillo_set_seed, 1},
    {"RcppArmadillo_fastLm", (DL_FUNC) &RcppArmadillo_fastLm, 2},
    {NULL, NULL, 0}
};

However, the function is now exported with the symbol name _RcppArmadillo_fastLm, not RcppArmadillo_fastLm, and so loading the package fails.

tl;dr: this line:

.Call( "RcppArmadillo_fastLm", X, y, PACKAGE = "RcppArmadillo" )

should become

.Call( "_RcppArmadillo_fastLm", X, y, PACKAGE = "RcppArmadillo" )

for compatibility with the newer version of Rcpp.

As an aside, it looks like fastLm is actually defined in two spots:

kevin@cdrv:~/r/pkg/RcppArmadillo [master]
$ ag "^fastLm <-"
R/fastLm.R
27:fastLm <- function(X, ...) UseMethod("fastLm")

R/RcppExports.R
4:fastLm <- function(X, y) {

but it looks like the version defined in fastLm.R 'wins' and becomes the actual version exported by RcppArmadillo. (Not sure if this is something that's already known.)

[binxiangni]

Yes, the same error message. I test it on Windows and the problem is the same. And every time I press check button on Rstudio, a message comes out:

==> Rcpp::compileAttributes()
* Updated src/RcppExports.cpp
* Updated R/RcppExports.R

which is consistent with what you mention.

[eddelbuettel]

Uh-oh. I see now. Unbelievable that did not hit us earlier.

What Kevin is of course correct, but there is an additional riddle. I had something similar moons ago in RProtoBuf. We have (had) symbols fastLm in both R/fastLm.R (as the default method) and in src/fastLm.cpp as an Rcpp export. That won't work under .registration=TRUE. They probably need to be distinct.

[eddelbuettel]

With that the call becomes a newer-school

    .Call(`_RcppArmadillo_fastLm_cpp`, X, y)

without the PACKAGE=... argument.

[eddelbuettel]

With a little delay due to the Saturday morning neighbourhood run, try this branch.

I renamed one of the two fastLm symbols as under .registration=TRUE we can only have one.

[binxiangni]

It works. @eddelbuettel @kevinushey Thanks for your time and quick remedy. Still not quite familiar with how the whole package magically runs, so I should spend time doing some research on it. Anyway, thanks for all your effort helping me.

[eddelbuettel]

That was a weird one pushed down on part by R 3.4.* (wanting registration of C-level routines, optionally with .registration=TRUE in file NAMESPACE) which leads to a few ugly side-effects like having to run compileAttributes() twice during transition. Bound to bite other people too.

Really surprised it did not bite any one of us til now. Oh well. Two pending PRs, plus change to configure for OpenMP detection to likely RcppArmadillo ".1" release next week.

[eddelbuettel]

It'll get closed automatically once the PR is merged.

[eddelbuettel]

Looked like I was wrong w.r.t. squashed merged. Learn something new.

PR #164 merged.

[dorianps]

Hi Dirk,

I use RcppArmadillo for one my packages. Travis builds used to work before, but now RcppArmadillo fails with a similar error as this one. Here is the log:
https://s3.amazonaws.com/archive.travis-ci.org/jobs/290835046/log.txt?X-Amz-Expires=29&X-Amz-Date=20171021T144353Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJRYRXRSVGNKPKO5A/20171021/us-east-1/s3/aws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=fae01b4332dac40fa06baacd054eaa7ab7912471ee87260883b21147948f6f8c

I see the current github version is 0.8.100.1.0.2 while the cran version is 0.8.100.1.0.

Is this indeed the problem that is causing my travis to fail? Any suggestion how to overcome it for now?

[eddelbuettel]

@dorianps: Please edit the question to make it useable.

Random links to random sites, AWS or not, don't count.

Note that we run reverse depends against all CRAN packages depending on RcppArmadillo and found no issues.

[dorianps]

Hi @eddelbuettel ,

Sorry, I thought the link was open for all.

Here is the error:

trying URL 'https://cloud.r-project.org/src/contrib/RcppArmadillo_0.8.100.1.0.tar.gz'
Content type 'application/x-gzip' length 1312598 bytes (1.3 MB)
==================================================
downloaded 1.3 MB

Installing RcppArmadillo
'/home/travis/R-bin/lib/R/bin/R' --no-site-file --no-environ --no-save  \
  --no-restore --quiet CMD INSTALL  \
  '/tmp/RtmpC2rkKa/devtools42884f7f92ce/RcppArmadillo'  \
  --library='/usr/local/lib/R/site-library' --install-tests 

* installing *source* package ‘RcppArmadillo’ ...
** package ‘RcppArmadillo’ successfully unpacked and MD5 sums checked
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether clang++ accepts -g... yes
checking how to run the C++ preprocessor... clang++ -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether clang++ accepts -g... (cached) yes
checking whether g++ version is sufficient... checking LAPACK_LIBS... fallback LAPACK from R 3.3.0 or later used
checking for macOS... not found as on Linux
checking for OpenMP... found
configure: creating ./config.status
config.status: creating inst/include/RcppArmadilloConfigGenerated.h
config.status: creating src/Makevars
** libs
clang++ -std=gnu++11 -I/home/travis/R-bin/lib/R/include -DNDEBUG  -I"/usr/local/lib/R/site-library/Rcpp/include" -I/home/travis/R-bin/include  -I../inst/include -fopenmp -fpic  -g -O2 -c RcppArmadillo.cpp -o RcppArmadillo.o
clang++ -std=gnu++11 -I/home/travis/R-bin/lib/R/include -DNDEBUG  -I"/usr/local/lib/R/site-library/Rcpp/include" -I/home/travis/R-bin/include  -I../inst/include -fopenmp -fpic  -g -O2 -c RcppExports.cpp -o RcppExports.o
clang++ -std=gnu++11 -I/home/travis/R-bin/lib/R/include -DNDEBUG  -I"/usr/local/lib/R/site-library/Rcpp/include" -I/home/travis/R-bin/include  -I../inst/include -fopenmp -fpic  -g -O2 -c fastLm.cpp -o fastLm.o
clang++ -std=gnu++11 -shared -L/home/travis/R-bin/lib/R/lib -L/home/travis/R-bin/lib -o RcppArmadillo.so RcppArmadillo.o RcppExports.o fastLm.o -fopenmp -L/home/travis/R-bin/lib/R/lib -lRlapack -L/home/travis/R-bin/lib/R/lib -lRblas -lgfortran -lm -lquadmath -L/home/travis/R-bin/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/RcppArmadillo/libs
** R
** inst
** tests
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error: package or namespace load failed for ‘RcppArmadillo’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/usr/local/lib/R/site-library/RcppArmadillo/libs/RcppArmadillo.so':
  libomp.so: cannot open shared object file: No such file or directory
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/usr/local/lib/R/site-library/RcppArmadillo’
Installation failed: Command failed (1)
missing: RcppArmadillo

Here is the full travis log:
log.txt

[coatless]

So, it looks like the compile is failing because you are missing libomp on your linux distro.

Try:

sudo apt-get install libiomp5

However, this might also be because your clang version is less than 3.7.0, which is the bare minimum for openmp with clang.

c.f. https://stackoverflow.com/questions/33357029/using-openmp-with-clang

[dorianps]

Thank's @coatless
Tried that but still does not work. Also tried to switch the ubuntu distro to the older Trusty, and that did not work either. Looks like libiomp5 is installed correctly but the .so library is still not found.

Here is the most recent travis log:
TravisLog_log_trusty_Oct21_lesymap.txt

In searching google for the error libomp.so: cannot open shared object file: No such file or directory there are a few posts from recent days, among which this one:
UCL/STIR#117

Yet, I am not so knowledgeable to understand what is wrong with openmp.

[eddelbuettel]

Well, I have obviously been using Rcpp since day one, and I have always been on Debian or Ubuntu. This never ever failed me for me. There is something "unusual" or hosed about your system.

Not an RcppArmadillo. Also, note that every Travis build is a full log of what works on a minimal Ubuntu 14.04 system. See the build logs here.

[eddelbuettel]

There is nothing wrong with libomp. It should always be installed as r-base-dev depends on build-essentials which pulls in a complete and working tool chain.

[dorianps]

I managed to fix the error by installing RcppArmadillo the good old way with install.packages early in the process. That uses g++ instead of clang, and things go smoothly. Turns out that one of my collaborators forces the use of clang in some of the scripts that are called later. Maybe those specific clang parameters create the problem, not sure. Here are the settings that were used, if anyone is curious:

export_clang_cxx() {
  echo "export CC=clang ;"
  echo "export CXX=clang++ ;"
  echo "export CXX1X=clang++ ;"
  echo "export CXX1XSTD=-std=c++11 ;"
  echo "export CXX11=clang++ ;"
}

clang_cxx() {
  mkdir -p ~/.R
  x=`clang --version`
  echo "clang --version returns: ${x}"
  x=`clang++ --version`
  echo "clang++ --version returns: ${x}"
  x=`export_clang_cxx`; eval ${x};
	CCACHER=""
	if [[ -n "${CCACHE}" ]]; then
		CCACHER="\$(CCACHE) "
		echo "CCACHE=${CCACHE}" >>  ~/.R/Makevars
	fi  
  echo "CC=${CCACHER}${CC}" >>  ~/.R/Makevars
  echo "CXX=${CCACHER}${CXX}" >> ~/.R/Makevars
  echo "CXX1X=${CCACHER}${CXX1X}" >> ~/.R/Makevars
  echo "CXX1XSTD=${CXX1XSTD}" >> ~/.R/Makevars
  echo "CXX11=${CCACHER}${CXX11}" >> ~/.R/Makevars
  # clang doesn't do well with openmp
  # if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then 
  #   echo "SHLIB_OPENMP_CXXFLAGS=" >> ~/.R/Makevars; 
  # fi  
}

Since my problem is resolved, the issue can be closed.

Thank you all.

[eddelbuettel]

There is nothing to be closed as it was alreday close but I am glad you took care of that self-inflicted wound.

[eddelbuettel]

@chitrak-banerjee Why are you adding this to a complete unrelated question? This makes no sense.

Please delete and open a fresh one with something reproducible.