Cannot sapply lambda functions

[romainfrancois]

With the cpp11 plugin on, Rcpp cannot sapply lambda functions.

#include <Rcpp.h>
using namespace Rcpp ;

// [[Rcpp:::plugins(cpp11)]]

// [[Rcpp::export]]
NumericVector double_up( NumericVector x ){
  return sapply( x, [](double x){ return x*x; } )
}

/*** R
  double_up( c(1,2) )
**/

I get :

$ RcppScript /tmp/double.cpp
In file included from double.cpp:1:
In file included from /Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include/Rcpp.h:27:
In file included from /Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include/RcppCommon.h:159:
In file included from /Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include/Rcpp/traits/traits.h:74:
/Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include/Rcpp/traits/result_of.h:31:22: error: no type named 'result_type' in
      '<lambda at double.cpp:8:21>'
        typedef typename T::result_type type ;
                ~~~~~~~~~~~~^~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include/Rcpp/sugar/functions/sapply.h:105:34: note: in instantiation of template class
      'Rcpp::traits::result_of<<lambda at double.cpp:8:21> >' requested here
        typename ::Rcpp::traits::result_of<Function>::type ,
                                 ^
double.cpp:8:10: note: while substituting deduced template arguments into function template 'sapply' [with RTYPE = 14, NA = true, T = Rcpp::Vector<14, PreserveStorage>,
      Function = <lambda at double.cpp:8:21>]
  return sapply( x, [](double x){ return x*x; } )
         ^
double.cpp:8:10: error: no matching function for call to 'sapply'
  return sapply( x, [](double x){ return x*x; } )
         ^~~~~~
/Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include/Rcpp/vector/ListOf.h:134:3: note: candidate template ignored: could not match 'ListOf' against
      'Vector'
T sapply(const ListOf<T>& t, Function fun) {
  ^
/Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include/Rcpp/sugar/functions/sapply.h:109:1: note: candidate template ignored: substitution failure
      [with RTYPE = 14, NA = true, T = Rcpp::Vector<14, PreserveStorage>, Function = <lambda at double.cpp:8:21>]
sapply( const Rcpp::VectorBase<RTYPE,NA,T>& t, Function fun ){
^
2 errors generated.
make: *** [double.o] Error 1
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include  -I"/Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rcpp/include"    -fPIC  -O3 -c double.cpp -o double.o
Erreur dans sourceCpp(tail(args, 1), verbose = "-v" %in% args) :
  Error 1 occurred building shared library.
Exécution arrêtée

This is mainly Rcpp::traits:: result_of's fault.

[eddelbuettel]

Oooof. Would be nice to have that working. Any idea of what to change / how to change it? Any chance you could cook up a PR?

[romainfrancois]

You need std::result_of from C++11 instead of the very simplistic Rcpp::traits::result_of.

Good luck with conditional compilation. I probably don't want to deal with that.

[eddelbuettel]

We have conditioned on 'do we have more than C++98' for many years within Rcpp, so there is no technical reason not to. Many other projects do as well as eg Armadillo. But yes, it is work, and as always with all of us being volunteers here this may or may not get done.

[romainfrancois]

I am well aware of these technicalities. I was just saying that I don't fancy being a part of it.
Being free from the burden of that kind of condition compilation was one of the several reason why Rcpp11 started.

[eddelbuettel]

Sure. But at the same time you also cut off a lot of users.

We just saw that with RcppArmadillo where the last version tickled a bug in g++-4.4 which, however you and I may feel about a version that old, is still used in a lot of places.

[MattPD]

Just a heads up, in case this may be of some use: It appears that patching result_of.h is not enough.

For testing purposes, I've experimented with replacing the code defining Rcpp::traits::result_of with the following:

#include <type_traits>
namespace Rcpp { namespace traits { using std::result_of; } }

This appears to work for sapply and a (regular) standalone function, say, sapply(x, g) given double g(double x) { return x; }.

However, there are still compilation errors for the Romain's lambda example:

result_of.cpp: In function 'Rcpp::NumericVector double_up(Rcpp::NumericVector)':
result_of.cpp:23:46: error: no matching function for call to 'sapply(Rcpp::NumericVector&, double_up(Rcpp::NumericVector)::<lambda(double)>)'
result_of.cpp:23:46: note: candidates are:
[...]/R/R-current/library/Rcpp/include/Rcpp/vector/ListOf.h:134:3: note: template<class T, class Function> T Rcpp::sapply(const Rcpp::ListOf<T>&, Function)
[...]/R/R-current/library/Rcpp/include/Rcpp/sugar/functions/sapply.h:109:1: note: template<int RTYPE, bool NA, class T, class Function> Rcpp::sugar::Sapply<RTYPE, NA, T, Function, Rcpp::traits::same_type<typename std::result_of<Function>::type, typename Rcpp::traits::storage_type<Rcpp::traits::r_sexptype_traits<typename std::result_of<Function>::type>::rtype>::type>::value> Rcpp::sapply(const Rcpp::VectorBase<RTYPE, LHS_NA, LHS_T>&, Function)

Not quite sure what's the cause here -- could it be that Rcpp::traits::r_sexptype_traits or Rcpp::traits::storage_type need some further adaptation?

In any case, if making sapply work with std::result_of is possible, then conditional compilation / dispatching (depending on C++11 support) groundwork has already been done by boost::result_of:

• http://www.boost.org/doc/libs/master/boost/utility/result_of.hpp
• http://www.boost.org/doc/libs/master/libs/utility/utility.htm#result_of

On a side note: It's header-only and already a part of the BH package. If it's relatively self-contained, then perhaps adding a dependency for this particular case wouldn't be too troubling (compared to the current status quo, i.e., not supporting lambdas at all)? In any case, there are only trade-offs...

[eddelbuettel]

Thanks for the follow-up. I have at times considers whether we'd want to depend on BH and it is tempting: header-only, (mostly) stable, well-tested, widely used, ... But it would add a pretty big package to all builds so it adds quite some weight. Pulling small things out of Boost is also an option, we already have a macro processor borrowed from it.

Do you feel like working some more on this?

[jjallaire]

In my experience it's hard to pull only "a little" out of boost. I think
once I tried to pull out "just" shared_ptr and I still ended up with
several megabytes of headers. So my 2 cents would be if you want to use
boost just use BH (but I still agree about being cautious about bringing BH
without at least thinking through all the implications/consequences). Note
that here is no end-user requirement of installing BH as it's only needed
at build time.

On Sat, May 2, 2015 at 11:43 AM, Dirk Eddelbuettel <notifications@github.com

wrote:

Thanks for the follow-up. I have at times considers whether we'd want to
depend on BH http://cran.rstudio.com/package=BH and it is tempting:
header-only, (mostly) stable, well-tested, widely used, ... But it would
add a pretty big package to all builds so it adds quite some weight.
Pulling small things out of Boost is also an option, we already have a
macro processor borrowed from it.

Do you feel like working some more on this?

—
Reply to this email directly or view it on GitHub
#213 (comment).

[eddelbuettel]

But build time is every Travis run for those working from source (rather than, say, my PPA). So BH would be expensive to some. At least until the caching gets better...

[jjallaire]

So they'd have to pay the BH download on every travis run. Not
insubstantial but hopefully also typically not costing much more than a few
seconds. I'm neutral on this overall (probably lean negative just to keep
Rcpp on the leaner side, however I wouldn't oppose doing it if there is a
strong user benefit).

On Sat, May 2, 2015 at 12:16 PM, Dirk Eddelbuettel <notifications@github.com

wrote:

But build time is every Travis run for those working from source (rather
than, say, my PPA
https://launchpad.net/%7Eedd/+archive/ubuntu/misc/+packages). So BH
would be expensive to some. At least until the caching gets better...

—
Reply to this email directly or view it on GitHub
#213 (comment).

[eddelbuettel]

I'd word it exactly the same way. On the margin, leaner Rcpp is better but we may get to a point where depending on BH will be a good idea. So let's keep the doors open...

[MattPD]

@eddelbuettel: can take a look, provided someone walks me through / gives hints on the encountered issue with Rcpp::traits::r_sexptype_traits / Rcpp::traits::storage_type :-) ATM it's not quite clear to me why these fail to cooperate with std::result_of (or even whether that's indeed what makes sapply fail with the lambda example, for that matter).

@jjallaire definitely makes a good point.

While Boost has been improving its modularization (since around 2013; cf. https://svn.boost.org/trac/boost/wiki/CMakeModularizationStatus & https://isocpp.org/blog/2013/11/boost-migrating) -- resulting in independent, library-specific repositories (here: https://github.com/boostorg/utility), at the same time I can see that boost/utility/result_of.hpp depends on Boost.Preprocessor, Boost.MPL, and others...

Some dependencies on Boost.MPL seem trivial / easily removable (AFAICT, it all boils down to the equivalents of C++11's std::true_type, std::false_type, and std::conditional -- and chances are these already have analogues in the Rcpp?). Others may warrant further investigation...

That being said, I think it's an "I cross that bridge when I come to it" matter at the moment for the issue at hand -- if cooperation with std::result_of is not possible / not sufficient, then discussing the best way to conditionally depend on it may be a bit premature ;-)

[kevinushey]

My opinions on this are somewhat mixed. I also prefer Rcpp being lean, but am open to bringing in Boost / BH if it has a large enough benefit. I am not sure if the benefit is large enough in this case.

One thing to worry about is, if Rcpp decides to depend on boost, it could break compilation on older compilers (e.g. if the release of Boost packaged in BH is somehow incompatible with an older GCC). One of the main drives for Rcpp to stick with C++98 is to ensure it can compile with such older compilers. (I could be wrong here -- I am not sure how committed Boost is in maintaining backwards compatibility with older compilers...)

At the same time, I feel that if a user really wants C++11 features, they should just use Rcpp11 or (if @romainfrancois plans to begin work on it soon) Rcpp14.

Finally, in my opinion, allowing lambdas to work on Rcpp types isn't a strong enough reason to bring BH. You can still use lambdas just fine on STL vectors, or even on an iterator pair constructed from an Rcpp vector, e.g.

std::for_each(x.begin(), x.end(), [](double x) { ... });

Of course, the semantics aren't quite the same as sapply but I do feel like it's 'good enough', at least for Rcpp...

[romainfrancois]

Depending on BH means each and every package that uses Rcpp will have to have

LinkingTo: Rcpp, BH

Not a huge deal but still.

Rcpp already has some conditional compilation on c++11, that can easily be used to work around this. And lambdas are c+11 anyway. So if you do want to support this C++11 feature, the best is to actually use C++11.

That should be easy enough, and you can easily borrow from Rpp11 or Rcpp14.

[jjallaire]

I didn't consider the fact that depending on BH would retroactively require all of our dependent packages to do LinkingTo: Rcpp, BH. That sounds like a complete nightmare to orchestrate, definitely not worth it given what we're talking about here (and also given Romain's point about conditional compilation being another way to approach this).