Crash in RStudio (not in R) when calling testFormat

[bobjansen]

I'm on:

> sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.1 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=nl_NL.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=nl_NL.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=nl_NL.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=nl_NL.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] tools_3.2.3

If I run

anytime:::testFormat("%m/%d/%Y", "22/3/2016")

RStudio (Version 0.99.903) crashes immediately but R ran from the command line does not.

My first hypothesis is that memory corruption happens in anytime in a way that only causes problems when the rsession is started like RStudio does and will try to confirm.

[eddelbuettel]

I tend to run much newer RStudio previews but they behave. Just to narrow this down as I have the same 64-bit platform: RStudio Server or Desktop?

[eddelbuettel]

I can replicate this. Will take a look.

[eddelbuettel]

@kevinushey I am at a loss. We have a hard crash with RStudio (including 1.0.106) with a piece of code that works fine on the command-line. As @bobjansen found, executing this in RStudio bombs.

I added a bunch of REprintf() but no luck. Narrowing down via #if 0 ... #endif lead to this line blowing things up. But that whole segment is a) from a Boost example, b) has worked fine in RcppBDT and this very package, anytime, in another function and c) does not come up via R -d gdb.

Is this possibly a bad Boost interaction? I am building via BH which is still at 1.60.0-2.

[eddelbuettel]

For reference here is the function as I currently have it, which includes two small and minor fixes relative to what GH has:

// [[Rcpp::export]]
Rcpp::NumericVector testFormat(const std::string fmt, const std::string s, const std::string tz = "") {

    REprintf("A0\n");
    bt::ptime pt, ptbase;

    REprintf("A1\n");
    std::istringstream is(s);
    REprintf("A2\n");
    std::locale loc = std::locale(std::locale::classic(), new bt::time_input_facet(fmt));
#if 0

    REprintf("A3\n");
    is.imbue(loc);
    is >> pt;
#endif
    REprintf("A4\n");
    double timeval = (pt == ptbase) ? NA_REAL : ptToDouble(pt);

    REprintf("A5\n");
    Rcpp::NumericVector pv(1);
    REprintf("A6\n");
    pv[0] = timeval;
    REprintf("A7\n");
    pv.attr("class") = Rcpp::CharacterVector::create("POSIXct", "POSIXt");
    pv.attr("tzone") = tz;
    REprintf("A8\n");

    return pv;
}

Moving the #if 0 one statement up makes it work fine (but of course return NA).

[kevinushey]

Interestingly, I'm not able to replicate this initially on OS X. I'll see if I can on an Ubuntu VM.

[eddelbuettel]

Yes, Bob and I are both on 16.04. I can try some RStudio Server installation on some of the blades here to see if it makes a difference.

[kevinushey]

I can replicate the crash. Do we have a rocker + RStudio container where we can run against a version of R + anytime build with sanitizers that could potentially shed more light here?

[eddelbuettel]

Rocker + Sanitizers, yes (now also via rhub (!!)) and I used that recently because this very package was UBSAN flagged.

But not with RStudio. But I guess you could just add the RStudio binary?

[eddelbuettel]

BTW the package itself is now UBSAN clean -- or so I think. The pages at CRAN are all from the 0.0.3 version and don't seem to have been updated to the now-current 0.0.4 version: https://cloud.r-project.org/web/checks/check_results_anytime.html

[eddelbuettel]

This looks really really gnarly. I moved a things around, tried via instantiation of the local TimeFormats class, but no luck.

What Boost version are you building against, @kevinushey ? Could that be the issue?

[kevinushey]

Here's what I see in RStudio with gdb attached to the R process -- some lovely STL template code...

Thread 1 "rsession" received signal SIGSEGV, Segmentation fault.
0x0000000000c25dfd in std::_Rb_tree<char, std::pair<char const, boost::date_time::string_parse_tree<char> >, std::_Select1st<std::pair<char const, boost::date_time::string_parse_tree<char> > >, std::less<char>, std::allocator<std::pair<char const, boost::date_time::string_parse_tree<char> > > >::_M_erase(std::_Rb_tree_node<std::pair<char const, boost::date_time::string_parse_tree<char> > >*) ()
(gdb) bt
#0  0x0000000000c25dfd in std::_Rb_tree<char, std::pair<char const, boost::date_time::string_parse_tree<char> >, std::_Select1st<std::pair<char const, boost::date_time::string_parse_tree<char> > >, std::less<char>, std::allocator<std::pair<char const, boost::date_time::string_parse_tree<char> > > >::_M_erase(std::_Rb_tree_node<std::pair<char const, boost::date_time::string_parse_tree<char> > >*)
    ()
#1  0x0000000000c260c8 in boost::date_time::date_input_facet<boost::gregorian::date, char, std::istreambuf_iterator<char, std::char_traits<char> > >::~date_input_facet() ()
#2  0x0000000000c26216 in boost::date_time::time_input_facet<boost::posix_time::ptime, char, std::istreambuf_iterator<char, std::char_traits<char> > >::~time_input_facet() ()
#3  0x00007f6a0e02ef54 in std::locale::_Impl::~_Impl() ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007f6a0e02f0f9 in std::locale::~locale() ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007f69e95d2cfc in std::basic_ios<char, std::char_traits<char> >::~basic_ios (this=0x7fffb0636f88, __in_chrg=<optimized out>)
    at /usr/include/c++/5/bits/basic_ios.h:282
#6  std::__cxx11::basic_istringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_istringstream (this=0x7fffb0636f10, 
    __in_chrg=<optimized out>, __vtt_parm=<optimized out>)
    at /usr/include/c++/5/sstream:433
#7  testFormat (fmt="%m/%d/%Y", s="22/3/2016", tz="") at anytime.cpp:292
#8  0x00007f69e95cedd2 in anytime_testFormat (fmtSEXP=<optimized out>, sSEXP=<optimized out>, 
    tzSEXP=<optimized out>) at RcppExports.cpp:50
#9  0x00007f6a0ee258b0 in ?? () from /usr/lib/R/lib/libR.so
#10 0x00007f6a0ee5bbff in Rf_eval () from /usr/lib/R/lib/libR.so
#11 0x00007f6a0ee5e0f8 in ?? () from /usr/lib/R/lib/libR.so
#12 0x00007f6a0ee5b9f1 in Rf_eval () from /usr/lib/R/lib/libR.so
#13 0x00007f6a0ee5d0b5 in Rf_applyClosure () from /usr/lib/R/lib/libR.so
#14 0x00007f6a0ee5b7cd in Rf_eval () from /usr/lib/R/lib/libR.so
#15 0x00007f6a0ee82ff2 in Rf_ReplIteration () from /usr/lib/R/lib/libR.so
#16 0x00007f6a0ee83371 in ?? () from /usr/lib/R/lib/libR.so
#17 0x00007f6a0ee83424 in run_Rmainloop () from /usr/lib/R/lib/libR.so
#18 0x0000000000d50702 in rstudio::r::session::runEmbeddedR(rstudio::core::FilePath const&, rstudio::core::FilePath const&, bool, bool, SA_TYPE, rstudio::r::session::Callbacks const&, rstudio::r::session::InternalCallbacks*) ()
#19 0x0000000000d31f8d in rstudio::r::session::run(rstudio::r::session::ROptions const&, rstudio::r::session::RCallbacks const&) ()
#20 0x000000000069ec99 in main ()

The error seems to indicate that the crash is occurring when attempting to construct a std::istringstream, which seems extra bizarre: https://github.com/eddelbuettel/anytime/blob/master/src/anytime.cpp#L292

[kevinushey]

My best hypothesis: some cross-talk between the version of Boost used by RStudio (a very old 1.50.0) and the version used by anytime (Boost current) is causing this crash? If so, that's not going to be an easy hole to get out of...

[kevinushey]

Also note that since RStudio links to Boost statically, there's almost surely some awkward cross-talk going on. This is a tough place to be since date_time is not a header-only library, so effectively it could be that anytime may be compiling against Boost 1.61.0, but linking to whatever's available at runtime (which, for rsession, may just be Boost 1.50.0)

[eddelbuettel]

Boost Date_time has been used header-only for several years now via the (underused) RcppBDT package. There is a #define one can use to not use input/output facilities -- and with that you get by without linking.

anytime does the same, and its core functionality worked so far. This one is a real stunner.

@jjallaire mentioned a trick to me some years ago about 'protecting' the namespace away via bcp. Maybe time to get serious about it?

[eddelbuettel]

If so, that's not going to be an easy hole to get out of...

How do I query from C++ code whether I am inside RStudio? I will for now just disable the function and return NA and a warning (maybe via `Rcpp::stop()).

[kevinushey]

The simplest way to check if you're within RStudio (or an RStudio sub-process) is by checking the RSTUDIO environment variable, e.g. ::getenv("RSTUDIO").

[eddelbuettel]

Does that work on Windoze?

[kevinushey]

Hmm, to my surprise checking ::getenv("RSTUDIO") does not work (even though Sys.getenv("RSTUDIO") reports it just fine). Maybe just call back to R's own Sys.getenv?

[eddelbuettel]

Yeah, just fought that for a moment in the Windows VM as well.

Using Sys.getenv() works. That is ugly as hell so I'll protect it inside the .onLoad() function we already have -- to preset a Rodney Dangerfield variable indicating that testFormat() should not run.

Could you register this issue internally? To the best of my knowledge this is the first time Boost has been biting our heads off.

[eddelbuettel]

Actually that makes everything easier because I just have to access a boolean in a local environment.

[eddelbuettel]

Ok, I have a fudge. I actually just renamed that C++ function by appending _impl and now check in an R function which calls it. Ugly, but works. No longer brings RStudio down.

[eddelbuettel]

@bobjansen Please give the current master branch a spin.

While it does not "fix" the underlying issue -- which is likely an extremely rare and hairy Boost interaction between two different pieces of software built with two different Boost versions, namely RStudio and anytime, it at least avoids taking down your machine.

[eddelbuettel]

Correction: It was still in a branch when I wrote this 30 minutes ago. Will merge to master now.

[bobjansen]

@eddelbuettel I gave current master a spin and it works as expected.

[eddelbuettel]

It is far from perfect -- but also the first (known to me) time that Boost is biting. We will have to keep an eye on it. Not crashing is a first step.

Thanks again for bringing this up.