Crash in RStudio (not in R) when calling testFormat #25

Closed
bobjansen opened this Issue Oct 24, 2016 · 25 comments

Projects

None yet

3 participants

@bobjansen
Contributor

I'm on:

> sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.1 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=nl_NL.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=nl_NL.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=nl_NL.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=nl_NL.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] tools_3.2.3

If I run

anytime:::testFormat("%m/%d/%Y", "22/3/2016")

RStudio (Version 0.99.903) crashes immediately but R ran from the command line does not.

My first hypothesis is that memory corruption happens in anytime in a way that only causes problems when the rsession is started like RStudio does and will try to confirm.

@bobjansen bobjansen changed the title from Crash in RStudio (not in R) when calling `testFormat` to Crash in RStudio (not in R) when calling testFormat Oct 24, 2016
@eddelbuettel
Owner

I tend to run much newer RStudio previews but they behave. Just to narrow this down as I have the same 64-bit platform: RStudio Server or Desktop?

@eddelbuettel
Owner

I can replicate this. Will take a look.

@eddelbuettel
Owner
eddelbuettel commented Oct 24, 2016 edited

@kevinushey I am at a loss. We have a hard crash with RStudio (including 1.0.106) with a piece of code that works fine on the command-line. As @bobjansen found, executing this in RStudio bombs.

I added a bunch of REprintf() but no luck. Narrowing down via #if 0 ... #endif lead to this line blowing things up. But that whole segment is a) from a Boost example, b) has worked fine in RcppBDT and this very package, anytime, in another function and c) does not come up via R -d gdb.

Is this possibly a bad Boost interaction? I am building via BH which is still at 1.60.0-2.

@eddelbuettel
Owner

For reference here is the function as I currently have it, which includes two small and minor fixes relative to what GH has:

// [[Rcpp::export]]
Rcpp::NumericVector testFormat(const std::string fmt, const std::string s, const std::string tz = "") {

    REprintf("A0\n");
    bt::ptime pt, ptbase;

    REprintf("A1\n");
    std::istringstream is(s);
    REprintf("A2\n");
    std::locale loc = std::locale(std::locale::classic(), new bt::time_input_facet(fmt));
#if 0

    REprintf("A3\n");
    is.imbue(loc);
    is >> pt;
#endif
    REprintf("A4\n");
    double timeval = (pt == ptbase) ? NA_REAL : ptToDouble(pt);

    REprintf("A5\n");
    Rcpp::NumericVector pv(1);
    REprintf("A6\n");
    pv[0] = timeval;
    REprintf("A7\n");
    pv.attr("class") = Rcpp::CharacterVector::create("POSIXct", "POSIXt");
    pv.attr("tzone") = tz;
    REprintf("A8\n");

    return pv;
}

Moving the #if 0 one statement up makes it work fine (but of course return NA).

@kevinushey

Interestingly, I'm not able to replicate this initially on OS X. I'll see if I can on an Ubuntu VM.

@eddelbuettel
Owner

Yes, Bob and I are both on 16.04. I can try some RStudio Server installation on some of the blades here to see if it makes a difference.

@kevinushey
kevinushey commented Oct 24, 2016 edited

I can replicate the crash. Do we have a rocker + RStudio container where we can run against a version of R + anytime build with sanitizers that could potentially shed more light here?

@eddelbuettel
Owner

Rocker + Sanitizers, yes (now also via rhub (!!)) and I used that recently because this very package was UBSAN flagged.

But not with RStudio. But I guess you could just add the RStudio binary?

@eddelbuettel
Owner

BTW the package itself is now UBSAN clean -- or so I think. The pages at CRAN are all from the 0.0.3 version and don't seem to have been updated to the now-current 0.0.4 version: https://cloud.r-project.org/web/checks/check_results_anytime.html

@eddelbuettel
Owner

This looks really really gnarly. I moved a things around, tried via instantiation of the local TimeFormats class, but no luck.

What Boost version are you building against, @kevinushey ? Could that be the issue?

@kevinushey

Here's what I see in RStudio with gdb attached to the R process -- some lovely STL template code...

Thread 1 "rsession" received signal SIGSEGV, Segmentation fault.
0x0000000000c25dfd in std::_Rb_tree<char, std::pair<char const, boost::date_time::string_parse_tree<char> >, std::_Select1st<std::pair<char const, boost::date_time::string_parse_tree<char> > >, std::less<char>, std::allocator<std::pair<char const, boost::date_time::string_parse_tree<char> > > >::_M_erase(std::_Rb_tree_node<std::pair<char const, boost::date_time::string_parse_tree<char> > >*) ()
(gdb) bt
#0  0x0000000000c25dfd in std::_Rb_tree<char, std::pair<char const, boost::date_time::string_parse_tree<char> >, std::_Select1st<std::pair<char const, boost::date_time::string_parse_tree<char> > >, std::less<char>, std::allocator<std::pair<char const, boost::date_time::string_parse_tree<char> > > >::_M_erase(std::_Rb_tree_node<std::pair<char const, boost::date_time::string_parse_tree<char> > >*)
    ()
#1  0x0000000000c260c8 in boost::date_time::date_input_facet<boost::gregorian::date, char, std::istreambuf_iterator<char, std::char_traits<char> > >::~date_input_facet() ()
#2  0x0000000000c26216 in boost::date_time::time_input_facet<boost::posix_time::ptime, char, std::istreambuf_iterator<char, std::char_traits<char> > >::~time_input_facet() ()
#3  0x00007f6a0e02ef54 in std::locale::_Impl::~_Impl() ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007f6a0e02f0f9 in std::locale::~locale() ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007f69e95d2cfc in std::basic_ios<char, std::char_traits<char> >::~basic_ios (this=0x7fffb0636f88, __in_chrg=<optimized out>)
    at /usr/include/c++/5/bits/basic_ios.h:282
#6  std::__cxx11::basic_istringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_istringstream (this=0x7fffb0636f10, 
    __in_chrg=<optimized out>, __vtt_parm=<optimized out>)
    at /usr/include/c++/5/sstream:433
#7  testFormat (fmt="%m/%d/%Y", s="22/3/2016", tz="") at anytime.cpp:292
#8  0x00007f69e95cedd2 in anytime_testFormat (fmtSEXP=<optimized out>, sSEXP=<optimized out>, 
    tzSEXP=<optimized out>) at RcppExports.cpp:50
#9  0x00007f6a0ee258b0 in ?? () from /usr/lib/R/lib/libR.so
#10 0x00007f6a0ee5bbff in Rf_eval () from /usr/lib/R/lib/libR.so
#11 0x00007f6a0ee5e0f8 in ?? () from /usr/lib/R/lib/libR.so
#12 0x00007f6a0ee5b9f1 in Rf_eval () from /usr/lib/R/lib/libR.so
#13 0x00007f6a0ee5d0b5 in Rf_applyClosure () from /usr/lib/R/lib/libR.so
#14 0x00007f6a0ee5b7cd in Rf_eval () from /usr/lib/R/lib/libR.so
#15 0x00007f6a0ee82ff2 in Rf_ReplIteration () from /usr/lib/R/lib/libR.so
#16 0x00007f6a0ee83371 in ?? () from /usr/lib/R/lib/libR.so
#17 0x00007f6a0ee83424 in run_Rmainloop () from /usr/lib/R/lib/libR.so
#18 0x0000000000d50702 in rstudio::r::session::runEmbeddedR(rstudio::core::FilePath const&, rstudio::core::FilePath const&, bool, bool, SA_TYPE, rstudio::r::session::Callbacks const&, rstudio::r::session::InternalCallbacks*) ()
#19 0x0000000000d31f8d in rstudio::r::session::run(rstudio::r::session::ROptions const&, rstudio::r::session::RCallbacks const&) ()
#20 0x000000000069ec99 in main ()

The error seems to indicate that the crash is occurring when attempting to construct a std::istringstream, which seems extra bizarre: https://github.com/eddelbuettel/anytime/blob/master/src/anytime.cpp#L292

@kevinushey

My best hypothesis: some cross-talk between the version of Boost used by RStudio (a very old 1.50.0) and the version used by anytime (Boost current) is causing this crash? If so, that's not going to be an easy hole to get out of...

@kevinushey

Also note that since RStudio links to Boost statically, there's almost surely some awkward cross-talk going on. This is a tough place to be since date_time is not a header-only library, so effectively it could be that anytime may be compiling against Boost 1.61.0, but linking to whatever's available at runtime (which, for rsession, may just be Boost 1.50.0)

@eddelbuettel
Owner

Boost Date_time has been used header-only for several years now via the (underused) RcppBDT package. There is a #define one can use to not use input/output facilities -- and with that you get by without linking.

anytime does the same, and its core functionality worked so far. This one is a real stunner.

@jjallaire mentioned a trick to me some years ago about 'protecting' the namespace away via bcp. Maybe time to get serious about it?

@eddelbuettel
Owner

If so, that's not going to be an easy hole to get out of...

How do I query from C++ code whether I am inside RStudio? I will for now just disable the function and return NA and a warning (maybe via `Rcpp::stop()).

@kevinushey
kevinushey commented Oct 24, 2016 edited

The simplest way to check if you're within RStudio (or an RStudio sub-process) is by checking the RSTUDIO environment variable, e.g. ::getenv("RSTUDIO").

@eddelbuettel
Owner

Does that work on Windoze?

@kevinushey

Hmm, to my surprise checking ::getenv("RSTUDIO") does not work (even though Sys.getenv("RSTUDIO") reports it just fine). Maybe just call back to R's own Sys.getenv?

@eddelbuettel
Owner

Yeah, just fought that for a moment in the Windows VM as well.

Using Sys.getenv() works. That is ugly as hell so I'll protect it inside the .onLoad() function we already have -- to preset a Rodney Dangerfield variable indicating that testFormat() should not run.

Could you register this issue internally? To the best of my knowledge this is the first time Boost has been biting our heads off.

@eddelbuettel
Owner

Actually that makes everything easier because I just have to access a boolean in a local environment.

@eddelbuettel
Owner

Ok, I have a fudge. I actually just renamed that C++ function by appending _impl and now check in an R function which calls it. Ugly, but works. No longer brings RStudio down.

@eddelbuettel
Owner

@bobjansen Please give the current master branch a spin.

While it does not "fix" the underlying issue -- which is likely an extremely rare and hairy Boost interaction between two different pieces of software built with two different Boost versions, namely RStudio and anytime, it at least avoids taking down your machine.

@eddelbuettel
Owner

Correction: It was still in a branch when I wrote this 30 minutes ago. Will merge to master now.

@bobjansen
Contributor

@eddelbuettel I gave current master a spin and it works as expected.

@eddelbuettel
Owner

It is far from perfect -- but also the first (known to me) time that Boost is biting. We will have to keep an eye on it. Not crashing is a first step.

Thanks again for bringing this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment