THREADED_COROUTINES: JSProc unit test fails

[danielmewes]

Something to look into at some point...

[ RUN      ] JSProc.EvalTimeout
Version: rethinkdb foo2 (debug) (CLANG 3.2 (tags/RELEASE_32/final))
error: Error in src/extproc/extproc_spawner.cc at line 157:
error: Guarantee failed: [instance == __null] 
error: Backtrace:
error: Mon Mar 31 15:04:56 2014

       1: rethinkdb_backtrace(void**, int) at rethinkdb_backtrace.cc:100
       2: backtrace_t::backtrace_t() at backtrace.cc:202
       3: lazy_backtrace_formatter_t::lazy_backtrace_formatter_t() at backtrace.cc:282
       4: format_backtrace(bool) at backtrace.cc:197
       5: report_fatal_error(char const*, int, char const*, ...) at errors.cc:83
       6: extproc_spawner_t::extproc_spawner_t() at extproc_spawner.cc:157
       7: JSProc_EvalTimeout_Test::TestBody() at jsproc.cc:10
       8: void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) at gtest.cc:1989
       9: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) at gtest.cc:2025
       10: testing::Test::Run() at gtest.cc:2061
       11: testing::TestInfo::Run() at gtest.cc:2237
       12: testing::TestCase::Run() at gtest.cc:2343
       13: testing::internal::UnitTestImpl::RunAllTests() at gtest.cc:4065
       14: bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) at gtest.cc:1989
       15: bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) at gtest.cc:2025
       16: testing::UnitTest::Run() at gtest.cc:3699
       17: RUN_ALL_TESTS() at gtest.h:2236
       18: main+0x41 at main.cc:8
       19: __libc_start_main+0xed at 0x7f1f91f7b76d (/lib/x86_64-linux-gnu/libc.so.6)
       20: ./rethinkdb-unittest() [0x2114cf9] at 0x2114cf9 ()

The same problem also affects the RDBInterrupt test.

[danielmewes]

To clarify: This does only affect the THREADED_COROUTINES built that we use for memory profiling, not the production versions.

[Tryneus]

This appears not to happen if you just run the JSProc unittests. Since this is failing on a line that is guaranteeing the extproc_pool_t is unique (as it is a singleton), I would suspect that another test is not being cleaned up properly. This could be a bug in the unittests or elsewhere, but I'm not too worried about it being a problem in the server, we only instantiate the extproc_pool_t in one place during startup.

[Tryneus]

If this is a cleanup problem in one test, it can be a little annoying to reproduce. The order that tests run in is determined when you compile, and I haven't found an easy way to change this order. Once you can reliably reproduce the error, you can do a binary search of tests to see which combination results in the error (using --gtest_filter).

[danielmewes]

@Tryneus sure, I can do that.

[danielmewes]

Here's the order. Let the search begin!

BarrierTest
BitsetTest
DBMTest
ClonePtrTest
PrintSecondary
ClusteringSuggester
DiskFormatTest
RPCSemilatticeTest
HashRegionTest
HTTPEscaping
DatumTest
LogMessageTest
OptionsTest
PerfmonTest
PrintfBufferTest
Http
RegionMap
UtilsTest
BtreeUtilsTest
VarintTest
BlobTest
BtreeMetainfo
BTreeSindex
JSON
UuidTest
VectorClock
JSProc

[danielmewes]

This is the smallest set of tests that I could find which still fails:
--gtest_filter=BTreeSindex.LowLevelOps:JSON.ArrayInsertDelete:JSProc.EvalTimeout
(in this order).

Wonder why the JSON one is necessary there. Maybe it just alters the timing?