3.3.1 release made jiebaR failed to compile

[alexwwang]

I met problem listed below when try to install jiebaR package on MS-r 3.3.1, Ubuntu16.04.
Neither CRAN nor devtools::install_github failed with the same error log no matter
which library(personal or system) path chosen.

Key error info：sh: I/usr/lib64/microsoft-r/3.3/lib64/R/include: No such file or directory

I've got in touch with developer of jieba and made sure it is raised by bugs of MS-r 3.3.1.
issue's here: jieba#42

As long as Makevars includes codes below the compiling process will raise error:

PKG_CPPFLAGS =   -I../inst/include 

CXX_STD = CXX11

another same issue raised of this bug: sparseHessianFD

error details：
* installing *source* package ‘jiebaR’ ...

** libs

I/usr/lib64/microsoft-r/3.3/lib64/R/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -DU_STATIC_IMPLEMENTATION -I"/usr/lib64/microsoft-r/3.3/lib64/R/library/Rcpp/include" -c RcppExports.cpp -o RcppExports.o

sh: I/usr/lib64/microsoft-r/3.3/lib64/R/include: No such file or directory

/usr/lib64/microsoft-r/3.3/lib64/R/etc/Makeconf:141: recipe for target 'RcppExports.o' failed

make: [RcppExports.o] Error 127 (ignored)

I/usr/lib64/microsoft-r/3.3/lib64/R/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -DU_STATIC_IMPLEMENTATION -I"/usr/lib64/microsoft-r/3.3/lib64/R/library/Rcpp/include" -c detect.cpp -o detect.o

sh: I/usr/lib64/microsoft-r/3.3/lib64/R/include: No such file or directory

/usr/lib64/microsoft-r/3.3/lib64/R/etc/Makeconf:141: recipe for target 'detect.o' failed

make: [detect.o] Error 127 (ignored)

I/usr/lib64/microsoft-r/3.3/lib64/R/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -DU_STATIC_IMPLEMENTATION -I"/usr/lib64/microsoft-r/3.3/lib64/R/library/Rcpp/include" -c get_idf.cpp -o get_idf.o

sh: I/usr/lib64/microsoft-r/3.3/lib64/R/include: No such file or directory

/usr/lib64/microsoft-r/3.3/lib64/R/etc/Makeconf:141: recipe for target 'get_idf.o' failed

make: [get_idf.o] Error 127 (ignored)

I/usr/lib64/microsoft-r/3.3/lib64/R/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -DU_STATIC_IMPLEMENTATION -I"/usr/lib64/microsoft-r/3.3/lib64/R/library/Rcpp/include" -c get_tuple.cpp -o get_tuple.o

sh: I/usr/lib64/microsoft-r/3.3/lib64/R/include: No such file or directory

/usr/lib64/microsoft-r/3.3/lib64/R/etc/Makeconf:141: recipe for target 'get_tuple.o' failed

make: [get_tuple.o] Error 127 (ignored)

gcc -std=gnu99 -I/usr/lib64/microsoft-r/3.3/lib64/R/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -DU_STATIC_IMPLEMENTATION -I"/usr/lib64/microsoft-r/3.3/lib64/R/library/Rcpp/include" -fpic -DU_STATIC_IMPLEMENTATION -O2 -g -c init.c -o init.o

I/usr/lib64/microsoft-r/3.3/lib64/R/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -DU_STATIC_IMPLEMENTATION -I"/usr/lib64/microsoft-r/3.3/lib64/R/library/Rcpp/include" -c segtype-v4.cpp -o segtype-v4.o

sh: I/usr/lib64/microsoft-r/3.3/lib64/R/include: No such file or directory

/usr/lib64/microsoft-r/3.3/lib64/R/etc/Makeconf:141: recipe for target 'segtype-v4.o' failed

make: [segtype-v4.o] Error 127 (ignored)

I/usr/lib64/microsoft-r/3.3/lib64/R/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -DU_STATIC_IMPLEMENTATION -I"/usr/lib64/microsoft-r/3.3/lib64/R/library/Rcpp/include" -c util.cpp -o util.o

sh: I/usr/lib64/microsoft-r/3.3/lib64/R/include: No such file or directory

/usr/lib64/microsoft-r/3.3/lib64/R/etc/Makeconf:141: recipe for target 'util.o' failed

make: [util.o] Error 127 (ignored)

I/usr/lib64/microsoft-r/3.3/lib64/R/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -DU_STATIC_IMPLEMENTATION -I"/usr/lib64/microsoft-r/3.3/lib64/R/library/Rcpp/include" -c word_freq.cpp -o word_freq.o

sh: I/usr/lib64/microsoft-r/3.3/lib64/R/include: No such file or directory

/usr/lib64/microsoft-r/3.3/lib64/R/etc/Makeconf:141: recipe for target 'word_freq.o' failed make: [word_freq.o] Error 127 (ignored)

-shared -L/usr/lib64/microsoft-r/3.3/lib64/R/lib -o jiebaR.so RcppExports.o detect.o get_idf.o get_tuple.o init.o segtype-v4.o util.o word_freq.o -L/usr/lib64/microsoft-r/3.3/lib64/R/lib -lR

sh: line 2: -shared: command not found

/usr/lib64/microsoft-r/3.3/lib64/R/share/make/shlib.mk:6: recipe for target 'jiebaR.so' failed

make: *** [jiebaR.so] Error 127

ERROR: compilation failed for package ‘jiebaR’

* removing ‘/home/da/R/x86_64-pc-linux-gnu-library/3.3/jiebaR’

Error: Command failed (1)

session_Info():
Session info ------------------------------------------------------------------------------------------------------------------
setting value
version R version 3.3.1 (2016-06-21)
system x86_64, linux-gnu
ui RStudio (0.99.902)
language (EN)
collate zh_CN.UTF-8
tz PRC
date 2016-10-26

Packages ----------------------------------------------------------------------------------------------------------------------
package * version date source
colorspace 1.2-6 2015-03-11 CRAN (R 3.3.0)
devtools * 1.12.0 2016-06-24 CRAN (R 3.3.1)
digest 0.6.9 2016-01-08 CRAN (R 3.2.5)
ggplot2 2.1.0 2016-03-01 CRAN (R 3.3.0)
gtable 0.2.0 2016-02-26 CRAN (R 3.3.0)
memoise 1.0.0 2016-01-29 CRAN (R 3.2.5)
munsell 0.4.3 2016-02-13 CRAN (R 3.3.0)
plyr 1.8.4 2016-06-08 CRAN (R 3.3.1)
Rcpp 0.12.5 2016-05-14 CRAN (R 3.3.0)
RevoUtils 10.0.1 2016-08-24 local
RevoUtilsMath * 10.0.0 2016-06-15 local
scales 0.4.0 2016-02-26 CRAN (R 3.3.0)
withr 1.0.2 2016-06-20 CRAN (R 3.3.1)

sessionInfo():
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.1 LTS

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=zh_CN.UTF-8 LC_COLLATE=zh_CN.UTF-8
[5] LC_MONETARY=zh_CN.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=zh_CN.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=zh_CN.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] devtools_1.12.0 RevoUtilsMath_10.0.0

loaded via a namespace (and not attached):
[1] colorspace_1.2-6 scales_0.4.0 plyr_1.8.4 RevoUtils_10.0.1 tools_3.3.1 withr_1.0.2 gtable_0.2.0
[8] rstudioapi_0.6 memoise_1.0.0 Rcpp_0.12.5 ggplot2_2.1.0 grid_3.3.1 digest_0.6.9 munsell_0.4.3

gcc edition: 5.4.0 20160609

[nathansoz]

To get jiebaR to install I removed the CXX_STD = CXX11 flag from the package and added the -std=c++11 to the PKG_CPPFLAGS line. I'll try to look at this a little bit more, but it seems that some of our compilation practices for compatibility are conflicting with the ability to set c++11 standard via the CXX_STD variable.

[nathansoz]

Found here:

http://r.789695.n4.nabble.com/CXX-STD-and-configure-ac-in-packages-td4687852.html

R discovers what features are available to it at compile time and subs in the correct flag (-std=c++0x or -std=c++11) based on the compiler that was used to compile R. The problem is that we are using gcc 4.1 as part of an attempt to make one build of MRO compatible with many distributions of Linux. I don't believe that gcc has any c++11 features, so R doesn't know what to do wtih the CXX_STD=CXX11 flag.

SLES11SP1 is our minimum viable Linux distribution that we need to support right now, so we could probably look at moving to that for MRO 3.3.3+. It has GCC 4.3, which includes support for C++0x and would have caused this error not to happen.

[alexwwang]

@nathansoz
Thank you for respond.
As I have learned from jiebaR that it requires gcc >= 4.9 to compile so I think maybe gcc 4.3 would possibly not help.
I am not a developer of jiebaR so I don't know exactly the details about what cxx11 features they used or why gcc>=4.9 is recommended.
Sorry for can't help on this issue but wish you could consider this situation.

[nathansoz]

Ah, but you are able to use gcc 4.9+ to compile binaries for use in packages even if we use an older GCC. The only issue here is that the CXX_STD flag does not work because of the compiler that we are currently using to compile R. When we move to gcc 4.3, the issue with the CXX_STD flag should go away and you shouldn't have any more issues (assuming you have gcc 4.9+ on whatever system you installed MRO on).

For now, the workaround is just to remove the CXX_STD line and add -std=c++11 to the PKG_CPPFLAGS line.

[qinwf]

On macOS, with -std=c++11,

error: invalid argument '-std=c++11' not allowed with 'C/ObjC'

Re-compiling jiebaR
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ  \
  --no-save --no-restore --quiet CMD INSTALL '/Users/qinwf/git/jiebaR'  \
  --library='/var/folders/4g/_hpd08x91gx6hhf7rr6dpw040000gn/T//Rtmpt1a42G/devtools_install_1b81dc72600'  \
  --no-R --no-data --no-help --no-demo --no-inst --no-docs --no-exec  \
  --no-multiarch --no-test-load --preclean 

During startup - Warning messages:

* installing *source* package 'jiebaR' ...
** libs
clang++ -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -std=c++11  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.3/Resources/library/Rcpp/include"   -fPIC  -Wall -mtune=core2 -g -O2  -c RcppExports.cpp -o RcppExports.o
clang++ -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -std=c++11  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.3/Resources/library/Rcpp/include"   -fPIC  -Wall -mtune=core2 -g -O2  -c detect.cpp -o detect.o
clang++ -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -std=c++11  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.3/Resources/library/Rcpp/include"   -fPIC  -Wall -mtune=core2 -g -O2  -c get_idf.cpp -o get_idf.o
clang++ -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -std=c++11  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.3/Resources/library/Rcpp/include"   -fPIC  -Wall -mtune=core2 -g -O2  -c get_tuple.cpp -o get_tuple.o
clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include -DLOGGING_LEVEL=LL_WARNING -std=c++11  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include -I"/Library/Frameworks/R.framework/Versions/3.3/Resources/library/Rcpp/include"   -fPIC  -Wall -mtune=core2 -g -O2  -c init.c -o init.o
error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
make: *** [init.o] Error 1
ERROR: compilation failed for package 'jiebaR'
* removing '/private/var/folders/4g/_hpd08x91gx6hhf7rr6dpw040000gn/T/Rtmpt1a42G/devtools_install_1b81dc72600/jiebaR'
Error: Command failed (1)
Execution halted

Exited with status 1.

-std=c++11 is not portable, and CRAN may reject packages contain this line.

[nathansoz]

Sorry, I wasn't trying to suggest that the jiebaR package maintainers make any changes to the package. I was simply suggesting that @alexwwang could make those modifications on a local copy of the package for the case of it not working on Ubuntu 16.04.

I've verified that the package works unmodified on OSX. The issue is Linux-only and should be resolved when we update our compilers.

[alexwwang]

Thank you both for your explanation and suggestion. @nathansoz @qinwf

[eddelbuettel]

The problem is that we are using gcc 4.1 as part of an attempt to make one build of MRO compatible with many distributions of Linux.

Are you sure you mean 4.1? Even 4.4 (as used in old RHEL) is way old now outlawed by, say, RcppArmadillo.

I strongly suggest you go with g++ or newer, and/or try to stick with the compilers R was configured with. On Linux, most people are used to builds from source and that is also what CRAN does for testing.

[nathansoz]

Are you sure you mean 4.1?

I do mean gcc 4.1, as this is the compiler that ships with RHEL/CentOS 5.11, which was our minimum viable OS until last release. We still need to support SLES 11 SP1 for now, so we can move to gcc 4.3.

I strongly suggest you go with g++ or newer

We are using gcc/g++ right now. Was there a specific version you wanted to recommend? I agree that we need to update the gcc/g++ version that we are using, but we have constraints due to the distributions we want to support.

On Linux, most people are used to builds from source

This is fine. We provide the source and people are welcome to build it. We produce binaries for distribution and there are two paths that can be followed to do so.

1. Produce a binary build for each distribution we would like to support.

2. Produce a binary build on a sufficiently old distribution that will run on most newer distributions.

We've chosen the latter path for various reasons, though we would be open to having a discussion about the merits of both options. It obviously creates issues such as this but I think there are several ways that we can work around them. I appreciate the feedback and would love any suggestions for how to better serve the community with the binary builds we are providing.

[nathansoz]

I'm also able to replace the following variables in $R_HOME/etc/Makeconf to get this to work on Ubuntu 16.04:

CXX1X = g++

CXX1XFLAGS = -O2 -g $(LTO)

CXX1XPICFLAGS = -fpic

CXX1XSTD = -std=c++11

I can add this in for our 3.3.2 release to add these variables on supported platforms (anything newer than CentOS 6).

[violaglenn]

Having the same issue with the most recent version of dbscan (see dbscan#13 above), but am not having success with the posted fixes. I've tried updating the Makevars file under src folder within dbscan_1.1-1.tar in the following ways:

Orignal Makevars: CXX_STD = CXX11
Fixes attempted:

1. CXX1X = g++

CXX1XFLAGS = -O2 -g $(LTO)

CXX1XPICFLAGS = -fpic

CXX1XSTD = -std=c++11

2. CXX1X=g++ -std=c++11
   CXX1XSTD=-std=c++11

3. PKG_CPPFLAGS = -I../inst/include/ -std=c++11

I then install via local file: install.packages(".../dbscan_1.1-1.tar.gz",repos=NULL, type="source")

Each returns the same errors.

Any thoughts?

[jeroenterheerdt]

Closing this issue as it is supposed to have been fixed.