Fix multithread CScheduler and reenable test

[paveljanik]

This fixes the deadlock in the CScheduler when there are other serviceQueues waiting for the new task to be added to the taskQueue, but schedule notifies only one of them who processes it and ends. And the other ones get stucked waiting for the new task which doesn't come in at all.

Hard to reproduce, see #6540 for the discussion.

Fixes #6540.

[fanquake]

Will also close #8005

[maflcko]

Tested ACK

28655b8:

$ for i in {1..10000}; do timeout 1 src/test/test_bitcoin --run_test=scheduler_tests &> /dev/null ;echo $? >> /tmp/res ;done;cat /tmp/res|sort|uniq -c
   9630 0
    370 124

70af981:

$ rm /tmp/res; for i in {1..10000}; do timeout 1 src/test/test_bitcoin --run_test=scheduler_tests &> /dev/null ;echo $? >> /tmp/res ;done;cat /tmp/res|sort|uniq -c
  10000 0

---

Travis issue unrelated.

[laanwj]

Thanks for fixing this!

[laanwj]

For some reason Travis is failing on this due to python zmq problems. I don't understand why, as there is no Python (or zmq) related change here. Will try to clear Travis' caches.

[paveljanik]

This is because my branch is based on the pre-python3 merge. Master is python3, thus there is no zmq module for python2...

[paveljanik]

Rebased should solve this, I think. Maybe travis kick can help now.

[paveljanik]

This helped. Hmm, shouldn't we temporary add python-zmq back to travis to workaround this?

Or should we modify the test script that requires python's zmq module to dot no anything when there is no such module installed (this could also simplify the tests setup for everyone!)? @MarcoFalke what do you think?

[maflcko]

@paveljanik python-zmq is already present in .travis.yml, you can't add it more than once ;)

I changed it (#7851) to fail instead, so errors are detected (and not silently ignored). The travis failure was an error due to broken cache. I think @laanwj cleared the cache and the travis issue is now fixed.

[maflcko]

ut re-ACK 166e4b0

[paveljanik]

@MarcoFalke there is python3-zmq, not python-zmq...

[theuni]

Does it need to signal in the catch/rethrow case as well?

[paveljanik]

@theuni I think rethrow case is there only for f throwing some exception. It is not a case for our multithread test. But in general, yes, I think there could be the same problem for exception throwing f.

[laanwj]

@paveljanik Are you going to fix that here?

[paveljanik]

No. I'm think about writing another test that will use the exception calls, but not here and not now, sorry.

[laanwj]

No need to be sorry, thanks for your fix! just needed clarity about which state this pull is in.

utACK 166e4b0