CCheckQueue Unit Tests #9497

JeremyRubin · 2017-01-09T19:04:40Z

This PR builds on #9495 to unit test the CCheckQueue for correctness.

The cases covered in these tests are:

Standard usage
Failing checks are caught
Prior blocks failing don't interfere with future blocks
No Memory leakage (all check destructors are called before new blocks allowed, memory is freed).
Thread Safety

instagibbs · 2017-01-09T19:09:29Z

src/test/checkqueue_tests.cpp

@@ -0,0 +1,395 @@
+// Copyright (c) 2012-2015 The Bitcoin Core developers


couple of years out of date :)

JeremyRubin · 2017-01-09T23:55:45Z

Sorry for the line noise; the earlier build error should be addressed now.

ryanofsky

Overall great and very clever tests (thread safety one was fun to figure out). Added a bunch of minor comments. The only two comments I would really urge you to consider are the ones on the Memory and FrozenCleanup tests. I think it would be good to check same conditions without allocating large chunks of memory or doing 1-second busy loops so these tests can be more efficient and more reliable.

ryanofsky · 2017-01-11T21:51:00Z

src/test/checkqueue_tests.cpp

+#include <unordered_set>
+#include <memory>
+#include "random.h"
+BOOST_FIXTURE_TEST_SUITE(checkqueue_tests, TestingSetup)


Maybe add a comment noting BasicTestingSetup can't be used because it doesn't set nScriptCheckThreads.

ryanofsky · 2017-01-11T21:51:42Z

src/test/checkqueue_tests.cpp

+ */
+void Correct_Queue_range(std::vector<size_t> range)
+{
+    auto small_queue = std::shared_ptr<Correct_Queue>(new Correct_Queue {128});


Maybe declare 128 and any other common parameters as constants above.

ryanofsky · 2017-01-11T21:56:34Z

src/test/checkqueue_tests.cpp

+            while (total) {
+                size_t r = GetRand(10);
+                std::vector<FakeCheckCheckCompletion> vChecks;
+                for (size_t k = 0; k < r && total; k++) {


Maybe replace the loop with vCheck.resize(min(total, r)).

ryanofsky · 2017-01-11T22:01:40Z

src/test/checkqueue_tests.cpp

+ */
+void Correct_Queue_range(std::vector<size_t> range)
+{
+    auto small_queue = std::shared_ptr<Correct_Queue>(new Correct_Queue {128});


Why is small_queue a shared pointer, not a unique pointer or just plain stack variable? Maybe add a comment explaining. Also, you could probably use make_shared if it does need to be a shared pointer.

ryanofsky · 2017-01-11T22:08:22Z

src/test/checkqueue_tests.cpp

+            }
+            result[end_fails ? 0 : 1] = control.Wait();
+        }
+        BOOST_CHECK(!result[0]);


Would seem more direct to just BOOST_CHECK the control.Wait() call instead of putting the results in an intermediate array. This way is ok too, though.

ryanofsky · 2017-01-11T22:11:31Z

src/test/checkqueue_tests.cpp

+    auto small_queue = std::shared_ptr<Correct_Queue>(new Correct_Queue {128});
+    boost::thread_group tg;
+    for (auto x = 0; x < nScriptCheckThreads; ++x) {
+       tg.create_thread([=]{small_queue->Thread();});


Replacing [=] with [&] might allow small_queue not to be a shared_ptr.

ryanofsky · 2017-01-11T22:17:24Z

src/test/checkqueue_tests.cpp

+    bool r = true;
+    for (size_t i = 0; i < COUNT; ++i)
+        r = r && UniqueCheck::results.count(i) == 1;
+    BOOST_REQUIRE(r);


Maybe also check that UniqueCheck::results.size == COUNT.

ryanofsky · 2017-01-11T22:18:15Z

src/test/checkqueue_tests.cpp

+
+// Test that blocks which might allocate lots of memory free their memory agressively.
+//
+// This test attempts to catch a pathalogical case where by lazily freeing


pathological (spelling)

ryanofsky · 2017-01-11T22:28:56Z

src/test/checkqueue_tests.cpp

+// checks might mean leaving a check un-swapped out, and decreasing by 1 each
+// time could leave the data hanging across a sequence of blocks.
+//
+// This test (failing) is dependent on not being able to handle


Instead of having the test be nondeterministic in this way, would anything be lost if you had the MemoryCheck constructor increment a static counter when passed a true arg, and the MemoryCheck destructor decrement the counter if the object was constructed with a true arg. Then you could detect the error case explicitly by checking the counter, and not have to allocate big chunks of memory.

ryanofsky · 2017-01-11T22:39:50Z

src/test/checkqueue_tests.cpp

+    //
+    // Note that this cannot cause a spurious failure, only could mean that
+    // the test doesn't actually end up checking that control waited.
+    MilliSleep(1000);


I think you could easily make this test deterministic as well, eliminating the long sleep and the while (frozen) busy loops. Would just need to have ~FrozenCleanupCheck increment a counter and signal a conditional variable so you could wait here for enough jobs to be frozen, and then do the boost check. Then this could signal another condition variable to unfreeze the jobs. I think it would be worth changing this to make the test more efficient and reliable.

ryanofsky

utACK d44af13ddc3615686a3e76cf9a3412999db0d692. Left one minor comment, feel free to ignore.

ryanofsky · 2017-01-13T22:58:32Z

src/test/checkqueue_tests.cpp

-        while (!done && !fails2) {
-            fails2 = queue->ControlMutex.try_lock();
+        std::mutex m;
+        bool has_lock {false};


It might be clearer to replace all these bools with an enum like { STARTED, TRY_LOCK, TRY_LOCK_DONE, DONE, DONE_ACK }.

Going to ignore this, I don't think it makes it more clear (to me, it's easier to debug several variables). If someone disagrees strongly, will change.

laanwj · 2017-01-19T15:32:12Z

ACK, needs squashing

JeremyRubin · 2017-01-19T17:07:08Z

Squashed!

JeremyRubin · 2017-01-20T22:19:59Z

Rebased to be on top of #9495.

…s constructors.

kallewoof · 2017-01-30T08:05:38Z

src/test/checkqueue_tests.cpp

+                control.Add(vChecks);
+            }
+        }
+        BOOST_REQUIRE(MemoryCheck::fake_allocated_memory == 0);


The MemoryCheck struct destructor does not --, so this should not be == 0 unless no MemoryCheck constructors are ever called.

Yes, it was the latter. The for loop never made anything (i = 9999; i<9999). Will fix :)

kallewoof · 2017-01-30T08:38:37Z

src/test/checkqueue_tests.cpp

+    };
+    ~MemoryCheck(){
+        if (b) {
+            fake_allocated_memory += 1;


fake_allocated_memory -= 1

kallewoof · 2017-01-30T08:43:30Z

src/test/checkqueue_tests.cpp

+BOOST_AUTO_TEST_CASE(test_CheckQueue_Recovers_From_Failure)
+{
+    auto fail_queue = std::unique_ptr<Failing_Queue>(new Failing_Queue {QUEUE_BATCH_SIZE});
+    std::array<FailingCheck, 100> checks;


Unused variable.

kallewoof · 2017-01-30T08:46:39Z

src/test/checkqueue_tests.cpp

+        // Wait until the queue has finished all jobs and frozen
+        FrozenCleanupCheck::cv.wait(l, [](){return FrozenCleanupCheck::nFrozen == 1;});
+        // Try to get control of the queue a bunch of times
+        for (auto x = 0; x < 100; ++x) {


Doing !fails && x < 100 here and simply fails = queue->ControlMutex.try_lock(); would break iteration on first fail rather than iterate over all 100 (e.g. if first try_lock() fails).

kallewoof · 2017-01-30T08:47:30Z

src/test/checkqueue_tests.cpp

+            // Wait for thread to get the lock
+            cv.wait(l, [&](){return has_lock;});
+            bool fails = false;
+            for (auto x = 0; x < 100; ++x) {


Same here with !fails && x < 100 as above.

JeremyRubin · 2017-02-16T06:42:08Z

Fixed the issues that @kallewoof raised, and squashed.

Unsquashed preserved here https://github.com/JeremyRubin/bitcoin/tree/checkqueue-tests-unsquashed.

kallewoof · 2017-02-25T17:58:22Z

utACK 96c7f2c

I'm a bit concerned about non-deterministic behavior in tests as this tends to be a pain when you do run into a problem. Or is this fixed seed / PRNG so that the numbers are always the same each time? (for GetRand())

JeremyRubin · 2017-02-25T19:22:04Z

I could make them deterministic if that's desirable, but realistically these tests are already non-deterministic by virtue of being multithreaded. None of the uses of GetRand are particularly dangerous here, although perhaps they area a little slower than could be.

kallewoof · 2017-02-25T21:31:16Z

I think that would be desirable, even if the multithreading makes it not 100%.

kallewoof · 2017-02-26T19:22:10Z

ACK 96c7f2c

96c7f2c Add CheckQueue Tests (Jeremy Rubin) e207342 Fix CCheckQueue IsIdle (potential) race condition and remove dangerous constructors. (Jeremy Rubin) Tree-SHA512: 5989743ad0f8b08998335e7ca9256e168fa319053f91b9dece9dbb134885bef7753b567b591acc7135785f23d19799ed7e6375917f59fe0178d389e961633d62

… into 0.14.2_fixes

96c7f2c Add CheckQueue Tests (Jeremy Rubin) e207342 Fix CCheckQueue IsIdle (potential) race condition and remove dangerous constructors. (Jeremy Rubin) Tree-SHA512: 5989743ad0f8b08998335e7ca9256e168fa319053f91b9dece9dbb134885bef7753b567b591acc7135785f23d19799ed7e6375917f59fe0178d389e961633d62

96c7f2c Add CheckQueue Tests (Jeremy Rubin) e207342 Fix CCheckQueue IsIdle (potential) race condition and remove dangerous constructors. (Jeremy Rubin)

bitcoin#9497

96c7f2c Add CheckQueue Tests (Jeremy Rubin) e207342 Fix CCheckQueue IsIdle (potential) race condition and remove dangerous constructors. (Jeremy Rubin) Tree-SHA512: 5989743ad0f8b08998335e7ca9256e168fa319053f91b9dece9dbb134885bef7753b567b591acc7135785f23d19799ed7e6375917f59fe0178d389e961633d62

…s constructors. zcash: cherry picked from commit e207342 zcash: bitcoin/bitcoin#9497

zcash: cherry picked from commit 96c7f2c zcash: bitcoin/bitcoin#9497

Bitcoin 0.15 locking PRs These are locking changes from upstream (bitcoin core) release 0.15, oldest to newest (when merged to the master branch). Each commit also includes a reference both to the PR and the upstream commit. - bitcoin/bitcoin#9497

instagibbs reviewed Jan 9, 2017

View reviewed changes

fanquake added the Tests label Jan 9, 2017

JeremyRubin force-pushed the checkqueue-tests branch 2 times, most recently from 641f5b8 to f4f1426 Compare January 9, 2017 22:48

ryanofsky reviewed Jan 11, 2017

View reviewed changes

JeremyRubin force-pushed the checkqueue-tests branch 2 times, most recently from 95316c2 to 4beddec Compare January 12, 2017 16:58

ryanofsky reviewed Jan 13, 2017

View reviewed changes

JeremyRubin force-pushed the checkqueue-tests branch from d44af13 to 747c5f3 Compare January 19, 2017 17:06

JeremyRubin force-pushed the checkqueue-tests branch from 747c5f3 to 5f84b90 Compare January 20, 2017 22:19

Fix CCheckQueue IsIdle (potential) race condition and remove dangerou…

e207342

…s constructors.

JeremyRubin force-pushed the checkqueue-tests branch from 5f84b90 to 0949835 Compare January 20, 2017 23:19

kallewoof reviewed Jan 30, 2017

View reviewed changes

src/test/checkqueue_tests.cpp Outdated

};

~MemoryCheck(){

if (b) {

fake_allocated_memory += 1;

Copy link

Member

kallewoof Jan 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fake_allocated_memory -= 1

kallewoof reviewed Jan 30, 2017

View reviewed changes

Add CheckQueue Tests

96c7f2c

JeremyRubin force-pushed the checkqueue-tests branch from 0949835 to 96c7f2c Compare February 16, 2017 06:38

JeremyRubin mentioned this pull request Mar 7, 2017

Lock-Free CheckQueue #9938

Closed

laanwj merged commit 96c7f2c into bitcoin:master Mar 14, 2017

paveljanik mentioned this pull request Mar 15, 2017

Wshadow: various gcc fixes #9911

Merged

JeremyRubin mentioned this pull request Mar 27, 2017

Fix CCheckQueue IsIdle (potential) race condition #9495

Closed

luke-jr added a commit to luke-jr/bitcoin that referenced this pull request Jun 5, 2017

Merge bitcoin#9497 via remote-tracking branch 'origin-pull/9497/head'…

1b864c9

… into 0.14.2_fixes

sickpig referenced this pull request in sickpig/BitcoinUnlimited Apr 27, 2018

Port Core #9497: CCheckQueue Unit Tests

e1e28f4

96c7f2c Add CheckQueue Tests (Jeremy Rubin) e207342 Fix CCheckQueue IsIdle (potential) race condition and remove dangerous constructors. (Jeremy Rubin)

michelvankessel added a commit to michelvankessel/blackcoin-more that referenced this pull request Jan 5, 2019

CCheckQueue Unit Tests bitcoin#9497

587bed7

bitcoin#9497

LarryRuane mentioned this pull request Feb 25, 2021

Bitcoin 0.15 locking PRs zcash/zcash#5012

Merged

LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Apr 2, 2021

Fix CCheckQueue IsIdle (potential) race condition and remove dangerou…

8e67b36

…s constructors. zcash: cherry picked from commit e207342 zcash: bitcoin/bitcoin#9497

LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Apr 2, 2021

Add CheckQueue Tests

a314cdb

zcash: cherry picked from commit 96c7f2c zcash: bitcoin/bitcoin#9497

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CCheckQueue Unit Tests #9497

CCheckQueue Unit Tests #9497

JeremyRubin commented Jan 9, 2017

instagibbs Jan 9, 2017

JeremyRubin Jan 9, 2017

JeremyRubin commented Jan 9, 2017

ryanofsky left a comment

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky Jan 11, 2017

ryanofsky left a comment

ryanofsky Jan 13, 2017

JeremyRubin Jan 16, 2017

laanwj commented Jan 19, 2017

JeremyRubin commented Jan 19, 2017

JeremyRubin commented Jan 20, 2017

kallewoof Jan 30, 2017 •

edited

Loading

JeremyRubin Feb 16, 2017

kallewoof Jan 30, 2017

kallewoof Jan 30, 2017

kallewoof Jan 30, 2017

JeremyRubin Feb 16, 2017

kallewoof Jan 30, 2017

JeremyRubin commented Feb 16, 2017

kallewoof commented Feb 25, 2017 •

edited

Loading

JeremyRubin commented Feb 25, 2017

kallewoof commented Feb 25, 2017

kallewoof commented Feb 26, 2017

		@@ -0,0 +1,395 @@
		// Copyright (c) 2012-2015 The Bitcoin Core developers

CCheckQueue Unit Tests #9497

CCheckQueue Unit Tests #9497

Conversation

JeremyRubin commented Jan 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JeremyRubin commented Jan 9, 2017

ryanofsky left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ryanofsky left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

laanwj commented Jan 19, 2017

JeremyRubin commented Jan 19, 2017

JeremyRubin commented Jan 20, 2017

kallewoof Jan 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JeremyRubin commented Feb 16, 2017

kallewoof commented Feb 25, 2017 • edited Loading

JeremyRubin commented Feb 25, 2017

kallewoof commented Feb 25, 2017

kallewoof commented Feb 26, 2017

kallewoof Jan 30, 2017 •

edited

Loading

kallewoof commented Feb 25, 2017 •

edited

Loading