Function name exported incorrectly in DLL, strange entries in tmp.def

[meowcat]

Hi,

I tried to compile the package https://github.com/khabbazian/sparseAHC/ under Windows. The package requires C++11 so I had to install the R devel build with gcc 4.9.3, and the latest Rtools.

I got compilation and installation to work using Rcpp (0.12.3, from CRAN source). Package loads fine. However, when I tried to use the functions:

• the Rcpp exported function sparseAHC_dgCIsSymmetric works correctly
• the Rcpp exported function sparseAHC_run_sparseAHC doesn't work.

I could not see anything wrong with the source files and therefore looked at the DLL with DependencyWalker. Interestingly:

• sparseAHC_dgCIsSymmetric is named correctly
• sparseAHC_run_sparseAHC is named
  `sparseAHC_run_sparseAHC.weak.ZNSt4listIiSaIiEE7emplaceIJiEEESt14_List_iteratorIiESt20_List_const_iteratorIiEDpOT._ZNK4Rcpp14not_compatible4whatEv.weak.ZNSt6vectorIdSaIdEE19_M_emplace_back_auxIJdEEEvDpOT._ZNK4Rcpp14not_compatible4whatEv.weak.ZNSt6vectorIiSaIiEE19_M_emplace_back_auxIJRKiEEEvDpOT._ZNK4Rcpp14not_compatible4whatEv.weak.ZNSt6vectorISt4pairIS0_IiiEdESaIS2_EE12emplace_backIJS2_EEEvDpOT._ZNK4Rcpp14not_compatible4whatEv.weak.ZNSt6vectorISt4pairIS0_IiiEdESaIS2_EE19_M_emplace_back_auxIJS2_EEEvDpOT._ZNK4Rcpp14not_compatible4whatEv.weak.ZNSt8_Rb_treeIiSt4pairIKiSt14_List_iteratorI7EdgeObjEESt10_Select1stIS5_ESt4lessIiESaIS5_EE22_M_emplace_hint_uniqueIJRKSt21piecewise_construct_tSt5tupleIJRS1_EESG_IJEEEEESt17_Rb_tree_iteratorIS5_ESt23_Rb_tree_const_iteratorIS5_EDpOT._ZNK4Rcpp14not_compatible4whatEv.weak.ZNSt8_Rb_treeIiSt4pairIKiSt14_List_iteratorIiEESt10_Select1stIS4_ESt4lessIiESaIS4_EE22_M_emplace_hint_uniqueIJRKSt21piecewise_construct_tSt5tupleIJOiEESF_IJEEEEESt17_Rb_tree_iteratorIS4_ESt23_Rb_tree_const_iteratorIS4_EDpOT._ZNK4Rcpp14not_compatible4whatEv`` instead!

To find out what is going on, I compiled again and captured the tmp.def which is generated during compilation. As one can see, directly behind the problematic function name there are a lot of entries starting with .weak that are apparently incorrectly picked up upon by the linker:

[...]
ZZN4Rcpp8internal12exitRNGScopeEvE3fun
 _ZZN4Rcpp8internal13enterRNGScopeEvE3fun
 sparseAHC_dgCIsSymmetric
 sparseAHC_run_sparseAHC
 .weak._ZNSt4listIiSaIiEE7emplaceIJiEEESt14_List_iteratorIiESt20_List_const_iteratorIiEDpOT_._ZNK4Rcpp14not_compatible4whatEv
 .weak._ZNSt6vectorIdSaIdEE19_M_emplace_back_auxIJdEEEvDpOT_._ZNK4Rcpp14not_compatible4whatEv
 .weak._ZNSt6vectorIiSaIiEE19_M_emplace_back_auxIJRKiEEEvDpOT_._ZNK4Rcpp14not_compatible4whatEv
 .weak._ZNSt6vectorISt4pairIS0_IiiEdESaIS2_EE12emplace_backIJS2_EEEvDpOT_._ZNK4Rcpp14not_compatible4whatEv
 .weak._ZNSt6vectorISt4pairIS0_IiiEdESaIS2_EE19_M_emplace_back_auxIJS2_EEEvDpOT_._ZNK4Rcpp14not_compatible4whatEv
 .weak._ZNSt8_Rb_treeIiSt4pairIKiSt14_List_iteratorI7EdgeObjEESt10_Select1stIS5_ESt4lessIiESaIS5_EE22_M_emplace_hint_uniqueIJRKSt21piecewise_construct_tSt5tupleIJRS1_EESG_IJEEEEESt17_Rb_tree_iteratorIS5_ESt23_Rb_tree_const_iteratorIS5_EDpOT_._ZNK4Rcpp14not_compatible4whatEv
 .weak._ZNSt8_Rb_treeIiSt4pairIKiSt14_List_iteratorIiEESt10_Select1stIS4_ESt4lessIiESaIS4_EE22_M_emplace_hint_uniqueIJRKSt21piecewise_construct_tSt5tupleIJOiEESF_IJEEEEESt17_Rb_tree_iteratorIS4_ESt23_Rb_tree_const_iteratorIS4_EDpOT_._ZNK4Rcpp14not_compatible4whatEv
 _Z12order_leavesRN5Eigen6MatrixIdLin1ELin1ELi0ELin1ELin1EEEi
 _Z13run_sparseAHCN5Eigen12SparseMatrixIdLi0EiEEN4Rcpp6VectorILi16ENS2_15PreserveStorageEEE
 _Z14dgCIsSymmetricN5Eigen12SparseMatrixIdLi0EiEEd
[...]

I cannot find this problem documented anywhere. But it seems that somehow additional exports are generated that start with .weak, and the linker mangles all of them into one function name.

Help?

[eddelbuettel]

I would suggest you try your luck on the r-devel list; none of us has g++-4.9 on Windows installed.

/cc @jeroenooms

[thirdwing]

Just take a look at the source code. Personally I think the only C++ feature used is the auto,
for example https://github.com/khabbazian/sparseAHC/blob/master/src/sparse_ahc.cpp#L66

Maybe you can replace it with the real types and have a try.

[jimhester]

I can reproduce this error using R-devel-493 and Rtools 3.3.

I can also confirm that removing the use of auto and compiling without the -std=c++11 and/or -std=c++0x flags remedies the situation. (See jimhester/sparseAHC@ddde20b)

However regardless of use of C++11 features compiling with -std=c++11 or -std=c++0x results in the observed error.

I finally tracked down the cause and remedy to this. As @meowcat mentions the .weak. references need to be removed from the nm output file. There is a related thread on r-devel where BDR notes he added a make variable NM_FILTER to change how the filtering is done (https://stat.ethz.ch/pipermail/r-devel/2014-July/069421.html).

So if you modify your Makeconf file (located at $(R_HOME}/etc$(R_ARCH)/Makeconf) to change the NM_FILTER from NM_FILTER = | sed -e '/.refptr./d' to NM_FILTER = | sed -e '/\.refptr\./d' -e '/\.weak\./d' it will also remove the .weak. references as well as the .refptr. references. Doing this and recompiling fixes the issue and the original code executes properly.

I am not sure if there are other types that could be output by nm which are not currently filtered, perhaps we should set NM_FILTER = | sed -e '/^ \./d' to pick up all of them...

[eddelbuettel]

Just wow.

Really nice detective work. Not really an Rcpp issue. I presume you will bring this to r-devel / R Core / thePowersThatBeWhoLookAfterRTools ?

[jimhester]

Yeah not an Rcpp issue at, I will try to get the default changed as I suspect this would crop up elsewhere and be fairly inscrutable to debug.

[thirdwing]

WOW~~

There seems still much work for the new tool chain.

[kevinushey]

Closing this as not an Rcpp issue -- thanks for digging in @jimhester!