Merge pull request #4380 from rakudo/workers-for-long-queues

Tweak worker addition for many small tasks

src/core.c/ThreadPoolScheduler.pm6

@@ -736,11 +736,22 @@ my class ThreadPoolScheduler does Scheduler {
             }
         }
 
-        # If we didn't complete anything, then consider adding more threads.
+        # Consider adding more worker threads when:
+        # 1. We didn't complete anything since the last supervision. This is
+        #    likely because some long-running tasks are holding onto the
+        #    workers. We should at least think about adding more (and the
+        #    code below will try to determine if that's beneficial).
+        # 2. The number of tasks in the queue is greater than the number of
+        #    workers. Such a situation suggests we are under-resourced, and
+        #    so liable to fall behind. Consider it like checkout lanes at a
+        #    supermarket: if 20 people are queueing and there are only 2 open
+        #    checkout lanes, it makes sense to open more, but if there are 20
+        #    people waiting and 30 open checkout lanes, there's little to be
+        #    won by opening another one at this point.
         my int $total-workers = nqp::elems($!general-workers)
           + nqp::elems($!timer-workers)
           + nqp::elems($!affinity-workers);
-        if $total-completed == 0 {
+        if $total-completed == 0 || nqp::elems(queue) > nqp::elems(worker-list) {
             if $total-workers < $!max_threads {
                 # There's something in the queue and we haven't completed it.
                 # If we are still below the CPU core count, just add a worker.