change in R-devel 4.0.0 breaks use of PKG_LIBS by plugins on Windows

[kevinushey]

The handling of command line arguments has changed subtly on Windows in R 4.0.0. Compare:

# R 3.6.2
> file.create("test.cpp")
[1] TRUE
> Rcpp::sourceCpp("test.cpp", dryRun = TRUE, rebuild = TRUE)
make cmd is
  make -f "C:/R/R-36~1.2PA/etc/x64/Makeconf" -f "C:/R/R-36~1.2PA/share/make/winshlib.mk" SHLIB_LDFLAGS='$(SHLIB_CXXLDFLAGS)' SHLIB_LD='$(SHLIB_CXXLD)' SHLIB="sourceCpp_4.dll" WIN=64 TCLBIN=64 OBJECTS="test.o"

make would use
C:/RBuildTools/3.5/mingw_64/bin/g++ -std=gnu++11  -I"C:/R/R-36~1.2PA/include" -DNDEBUG   -I"C:/Users/kevinushey/r/win-library/3.6/Rcpp/include" -I"C:/tmp/RtmpyOYaAF"        -O2 -Wall  -mtune=core2 -c test.cpp -o test.o
if test "ztest.o" != "z"; then \
  if test -e "sourceCpp_4-win.def"; then \
    echo C:/RBuildTools/3.5/mingw_64/bin/g++ -std=gnu++11  -shared -s -static-libgcc -o sourceCpp_4.dll sourceCpp_4-win.def test.o    -L"C:/R/R-36~1.2PA/bin/x64" -lR ; \
    C:/RBuildTools/3.5/mingw_64/bin/g++ -std=gnu++11  -shared -s -static-libgcc -o sourceCpp_4.dll sourceCpp_4-win.def test.o    -L"C:/R/R-36~1.2PA/bin/x64" -lR ; \
  else \
    echo EXPORTS > tmp.def; \
    C:/RBuildTools/3.5/mingw_64/bin/nm test.o | sed -n 's/^.* [BCDRT] / /p' | sed -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' | sed 's/[^ ][^ ]*/"&"/g'  >> tmp.def; \
    echo C:/RBuildTools/3.5/mingw_64/bin/g++ -std=gnu++11  -shared -s -static-libgcc -o sourceCpp_4.dll tmp.def test.o    -L"C:/R/R-36~1.2PA/bin/x64" -lR ; \
    C:/RBuildTools/3.5/mingw_64/bin/g++ -std=gnu++11  -shared -s -static-libgcc -o sourceCpp_4.dll tmp.def test.o    -L"C:/R/R-36~1.2PA/bin/x64" -lR ; \
    rm -f tmp.def; \
  fi \
fi


# R 4.1.0 devel
> setwd(tempdir())
> file.create("test.cpp")
[1] TRUE
> Rcpp::sourceCpp("test.cpp", dryRun = TRUE, rebuild = TRUE)            
make cmd is
  make -f "C:/R/R-devel/etc/x64/Makeconf" -f "C:/R/R-devel/share/make/winshlib.mk" SHLIB_LDFLAGS='$(SHLIB_CXXLDFLAGS)' SHLIB_LD='$(SHLIB_CXXLD)' SHLIB="sourceCpp_3.dll" PKG_LIBS='' WIN=64 TCLBIN=64 OBJECTS="test.o"

make would use
c:/Rtools/mingw_64/bin/g++ -std=gnu++11  -I"C:/R/R-devel/include" -DNDEBUG   -I"C:/R/R-devel/library/Rcpp/include" -I"C:/tmp/Rtmp8epDhl"   -I"C:/projects/BUILD/R-source-win64/extsoft/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c test.cpp -o test.o
if test "ztest.o" != "z"; then \
  if test -e "sourceCpp_3-win.def"; then \
    echo c:/Rtools/mingw_64/bin/g++ -std=gnu++11  -shared -s -static-libgcc -o sourceCpp_3.dll sourceCpp_3-win.def test.o  -L"C:/projects/BUILD/R-source-win64/extsoft/lib/x64" -L"C:/projects/BUILD/R-source-win64/extsoft/lib"  -L"C:/R/R-devel/bin/x64" -lR ; \
    c:/Rtools/mingw_64/bin/g++ -std=gnu++11  -shared -s -static-libgcc -o sourceCpp_3.dll sourceCpp_3-win.def test.o  -L"C:/projects/BUILD/R-source-win64/extsoft/lib/x64" -L"C:/projects/BUILD/R-source-win64/extsoft/lib"  -L"C:/R/R-devel/bin/x64" -lR ; \
  else \
    echo EXPORTS > tmp.def; \
    c:/Rtools/mingw_64/bin/nm test.o | sed -n 's/^.* [BCDRT] / /p' | sed -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' | sed 's/[^ ][^ ]*/"&"/g'  >> tmp.def; \
    echo c:/Rtools/mingw_64/bin/g++ -std=gnu++11  -shared -s -static-libgcc -o sourceCpp_3.dll tmp.def test.o  -L"C:/projects/BUILD/R-source-win64/extsoft/lib/x64" -L"C:/projects/BUILD/R-source-win64/extsoft/lib"  -L"C:/R/R-devel/bin/x64" -lR ; \
    c:/Rtools/mingw_64/bin/g++ -std=gnu++11  -shared -s -static-libgcc -o sourceCpp_3.dll tmp.def test.o  -L"C:/projects/BUILD/R-source-win64/extsoft/lib/x64" -L"C:/projects/BUILD/R-source-win64/extsoft/lib"  -L"C:/R/R-devel/bin/x64" -lR ; \
    rm -f tmp.def; \
  fi \
fi

In short: on R-devel, PKG_LIBS='' now gets set during compilation when Rcpp::sourceCpp() is called, which means that plugins which rely on setting PKG_LIBS (e.g. RcppParallel) will fail to link, as the requisite library paths will not be passed along.

You can see an example of these linker errors in RcppParallel right now here:

https://www.r-project.org/nosvn/R.check/r-devel-windows-ix86+x86_64-gcc8/RcppParallel-00check.html

Rcpp can accommodate this by ensuring no empty string gets passed into the command generated -- in particular, here:

https://github.com/RcppCore/Rcpp/blob/master/R/Attributes.R#L137-L138

I believe we should only provide this argument if we really do have a dependency source path.

(I suspect that wch/r-source@e836ed8 is the relevant change in R-devel that is affecting us here)

I will see if I can prepare an appropriate patch.

[eddelbuettel]

I see the difference you highlight (with a lot of squinting).

So another layer and 'test for empty' on lines 137+138 there?

(BTW Your title refers to R 4.0.0, your r-devel already has R 4.1.0 -- same for my week(end)ly automated builds of rocker/drd and alike: those 'jumped' and are at R 4.1.0.)

(And these architecture-dependent bugs really are a flavour of hell I could do without these days...)

[coatless]

Any chance of getting this fixed upstream in R?

[eddelbuettel]

Timing is less than ideal with R 4.0.0 due in 3+ weeks.

If you produce obe and reason well with then maybe you can get in patch in. Your chance to shine 😀

[eddelbuettel]

Also, R 4.0.0 broke LAPACK on Windows again for RcppArmadillo so that is hold now 😞

[jeroen]

@eddelbuettel that looks very much like the same problem as RcppParallel. It could be the same bug.

[jeroen]

FYI this is not a bug in base R; a fix in base-r has revealed a bug in Rcpp.

[coatless]

@jeroen thanks for clarifying!

[eddelbuettel]

See the bug just filed by @kevinushey on R's bugzilla which looks at the underlying change (as far as I can tell). So maybe let's call it 'unfortunate interaction' triggered by changes in R 4.0.0. But something we could (and have, thanks @kevinushey) fixed in Rcpp. And indeed a most helpful hint by @jeroen which should also (eventually) help RcppArmadillo which was also affected but where I did not see the root cause as clearly as @kevinushey.