nloptr 2.0.0: Fails to install from source on Ubuntu 18.04.6

[HenrikBengtsson]

Hi. FYI, installing nloptr 2.0.0 and its dependencies from source appears to fail on an up-to-date Ubuntu 18.04.6 system.

This might be because cmake 3.10.2, which is (>= 2.7.0), does not support option -j.

Details:

$ dpkg -l libnlopt0
dpkg-query: no packages found matching libnlopt0

$ dpkg -l libnlopt-dev
dpkg-query: no packages found matching libnlopt-dev

$ command -v cmake
/usr/bin/cmake
$ cmake --version
cmake version 3.10.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).

$ command -v gcc
/usr/bin/gcc
$ gcc --version
gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ Rscript --version
R scripting front-end version 4.1.2 (2021-11-01)

$ R --vanilla

R version 4.1.2 (2021-11-01) -- "Bird Hippie"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
...

> chooseCRANmirror(ind = 1)
> install.packages("nloptr")
Installing package into ‘/home/hb/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/nloptr_2.0.0.tar.gz'
Content type 'application/x-gzip' length 2219414 bytes (2.1 MB)
==================================================
downloaded 2.1 MB

* installing *source* package ‘nloptr’ ...
** package ‘nloptr’ successfully unpacked and MD5 sums checked
** using staged installation
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=gnu++14 accepts -g... yes
checking how to run the C++ preprocessor... g++ -std=gnu++14 -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ -std=gnu++14 accepts -g... (cached) yes
checking for pkg-config... /usr/bin/pkg-config
checking if pkg-config knows NLopt... no
using NLopt via local cmake build on x86_64 
set CMAKE_BIN=/usr/bin/cmake
set CC=gcc
set CFLAGS=-I/usr/local/include -fpic -g -O2
set CXX=g++
set CXXFLAGS=-std=gnu++11 -I/usr/local/include -fpic -g -O2
set LDFLAGS=-L/usr/local/lib
CMake Error: The source directory "/tmp/hb/RtmpG9S82r/R.INSTALL55019f8d6af/nloptr/src/nlopt-build" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
Unknown argument -j
Unknown argument 2
Usage: cmake --build <dir> [options] [-- [native-options]]
Options:
  <dir>          = Project binary directory to be built.
  --target <tgt> = Build <tgt> instead of default targets.
                   May only be specified once.
  --config <cfg> = For multi-configuration tools, choose <cfg>.
  --clean-first  = Build target 'clean' first, then build.
                   (To clean only, use --target 'clean'.)
  --use-stderr   = Ignored.  Behavior is default in CMake >= 3.0.
  --             = Pass remaining options to the native tool.
CMake Error: The source directory "/tmp/hb/RtmpG9S82r/R.INSTALL55019f8d6af/nloptr/src/nlopt" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
cp: cannot stat 'nlopt/include/*': No such file or directory
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gcc -I"/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/include" -DNDEBUG -I../inst/include  -I'/home/hb/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c init_nloptr.c -o init_nloptr.o
gcc -I"/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/include" -DNDEBUG -I../inst/include  -I'/home/hb/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c nloptr.c -o nloptr.o
g++ -std=gnu++11 -I"/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/include" -DNDEBUG -I../inst/include  -I'/home/hb/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c test-C-API.cpp -o test-C-API.o
g++ -std=gnu++11 -I"/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/include" -DNDEBUG -I../inst/include  -I'/home/hb/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c test-runner.cpp -o test-runner.o
g++ -std=gnu++11 -shared -L/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/lib -L/usr/local/lib -o nloptr.so init_nloptr.o nloptr.o test-C-API.o test-runner.o -L/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/lib -lRlapack -L/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/lib -lRblas -lgfortran -lm -lquadmath -Lnlopt/lib -lnlopt -L/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/lib -lR
/usr/bin/ld: cannot find -lnlopt
collect2: error: ld returned 1 exit status
/home/hb/shared/software/CBI/R-4.1.2-gcc8/lib/R/share/make/shlib.mk:10: recipe for target 'nloptr.so' failed
make: *** [nloptr.so] Error 1
ERROR: compilation failed for package ‘nloptr’
* removing ‘/home/hb/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8/nloptr’

The downloaded source packages are in
	‘/tmp/hb/RtmpzMvYSa/downloaded_packages’
Warning message:
In install.packages("nloptr") :
  installation of package ‘nloptr’ had non-zero exit status
> 

PS. I can probably figure this out myself. Just sharing as an FYI.

[eddelbuettel]

Install libnlopt directly (in a current enough version) and cmake is no longer needed.

[HenrikBengtsson]

Install libnlopt directly (in a current enough version) and cmake is no longer needed.

Thanks. Yes, I understood that as a workaround, but wanted to share that the everything-from-source version is not working.

[eddelbuettel]

Yes, @astamm is learning step by step that simply saying 'hey my package requires cmake' does not solve all problems all the time....

Also, "everything from source" is working to the satisfaction of CRAN, so (in shorthand notation) "is working". Your choice of combining a four-year old OS release with a new / current CRAN package simply puts up new challenges for you that we do not promise to have cleared. (I understand where you come from with ancient HPC setups etc; @astamm as a CNRS employee is probably sympathetic but "we all" just have to draw some lines somewhere.)

[astamm]

Thanks for sharing the issue.
As a side note, the system requirement is cmake but the version >= 2.7.0 refers to nlopt. I will make that clearer in a future release.
Also, I might switch to using one core only to compile nlopt, avoiding at all the -j flag. After all, it is pretty quick to compile on the fly wrt other external libraries.
Or I might put a requirement on the version number of cmake that is required. The 3.10 is far behind the newest 3.22.

[HenrikBengtsson]

@eddelbuettel, please - it's counterproductive to talk down to users running on older Linux distros (still in LTS), especially since there are lots of users in such a situation. Such comments risk preventing others from reporting similar problems they experience in the future. I personally have the skill and the options run with modern versions, and I do, but that's not the issue, or why I reported this here. I reported this here because I know there are 1,000's of users out there on these distros, and they might be struggling at this very moment, and if they don't have the experience or the skills, these error messages can be major blockers. It's also clear that the maintainer is not on this platform, because then they'd noticed. So, I am just trying to help all sides here. By reporting this problem here, others might pitch in, e.g. by identifying a better lower bound of CMake than 2.7.0 that @astamm can specify in the DESCRIPTION. That alone, might spare lots of hours across the world. I could have chosen not to spend 15 minutes troubleshooting and writing this up to share my experience here.

[HenrikBengtsson]

Thanks, @astamm. All your alternatives sounds good to me.

[astamm]

I am very grateful indeed @HenrikBengtsson. I am indeed more of a Mac user and I admit I had not thought of people using too old versions of cmake. I think the option of putting a version number requirement on cmake is safer because, who knows, if someone uses a version older than yours, there might be other issues he could face. On the other hand, I could lower down this requirement at least at the version of cmake you currently use if I switch to always using one core. Hmm, I'll think about the options and implement one in the next release. And I will definitely make the SystemRequirements field clearer. Thanks again for taking the time, really.

[eddelbuettel]

@astamm To be perfectly honest one thing that would really help is to not rely on cmake in the first place. That would have prevented #99 yesterday as well. As discussed, maybe use it create a Makefile (or set of Makefiles) and ship those. I am sure @HenrikBengtsson will be glad to be of assistance on his platform to test that setup. What we all have going for us here is that a) NLopt does not demand much and b) neither does R. But everthing can (and will) get in the way because invariably you will get another user who thinks you should also support Ubuntu 16.04, or CentOS 5.*, or ...

[astamm]

Always happy with honesty :-) That could be a way too as we discussed indeed. I kind of like the idea of cmake which stands for cross-platform make and is supposed to make cross-platform availability easier. And I think that people running on too old versions of cmake (where too old is wrt the use we are making of it) should simply be informed that if they want to be able to use the nloptr package then they either need a recent enough version of cmake (>= 3.sthg) or a recent enough system build of nlopt (>=2.7.0). I understand why some people sometimes need to stay on older versions of things but I'd rather encourage to install at least some recent enough versions. Of course, that means we might lose some current nloptr users. But they might come back later as well.

[eddelbuettel]

No need to explain cmake to me, we use it at my day job, and I see it in many other projects. What I have been trying (and failing) to get across is that it is not common on R build systems. Hence your long dialogue with Tomas. Hence #99.

It's complicated. For my $dayjob package I now have to rely on C++17. Which is, quite frankly, awesome for being able to use regexps and file system modules and whatnot but a pain for code builds. We also depend on a much newer cmake so I try hard to not let the R package at CRAN require using it. It's all about tradeoffs. And customs. And R is a GNU project to autoconf / configure is quite simply the standard so far. But hey you got Prof Ripley to add an entire section on cmake to the manual so who know in a decade or two you may be the one standing here and smiling

[eddelbuettel]

@HenrikBengtsson I re-read what I wrote. There is not talking down here anywhere. All I continue to say is 'works at CRAN, we offer help here as volunteers'. If you use a non-CRAN platform, the excitement and adventure is all yours. I look forward to many more such reports.

[HenrikBengtsson]

TL;DR: I think the requirement should be cmake (>= 3.15.0).

It looks like cmake gained support for -j ${NCORES} in version 3.13.0.

However, trying with cmake 3.13.0-3.14.5 gives:

> install.packages("nloptr")
...
[ 96%] Linking CXX static library libnlopt.a
[100%] Built target nlopt
CMake Error: The source directory "/home/hb/repositories/other/nloptr/src/nlopt" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
cp: cannot stat 'nlopt/include/*': No such file or directory
configure: creating ./config.status
config.status: creating src/Makevars
...
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (nloptr)

So, still an error, but install.packages() doesn't realize it.

Using cmake 3.15.0, or newer, works!

(*) Building from source (obviously) requires nlopt.h on the system, which on Ubuntu is provided by sudo apt install libnlopt-dev.

[astamm]

Thanks @HenrikBengtsson. I will add this to the SystemRequirements field.