Fixed size ThreadPool scheduler

[maierlars]

Scope & Purpose

This PR implements a new ThreadPoolScheduler with a fixed number of threads, and a number of different thread pools. The ThreadPoolScheduler creates a separate thread pool for each priority lane.

We have the following thread pools:

• SimpleThreadPool - this is a very simple thread pool with a std::queue, mutex + condition variable
• LockFreeThreadPool - this one is similar to the SimpleThreadPool but replaces the std::queue with a boost::lockfree::queue
• WorkStealingThreadPool - this is a first version of a work stealing pool where instead of a globally shared queue, each thread has a local queue

This PR also implements a number of synthetic benchmarks as tests (by default these are disabled so that they do not run durng regular builds). These benchmarks showed, that the WorkStealingThreadPool scales significantly better than the other two thread pools, and also much better then the current SupervisedScheduler.

[goedderz]

Soo much simpler 😍

[jsteemann]

LGTM

[jsteemann]

Note that the queue fill grade may be checked by a load balancer periodically to determine if the instance is still in good shape.
Consider this calling code, which determines the instance's availability for further requests, from a load balancer's perspective:

        // if the scheduler's queue is more than x% full, render
        // the server unavailable
        double unavailabilityFillGrade =
            scheduler->unavailabilityQueueFillGrade();
        if (unavailabilityFillGrade > 0.0) {
          double fillGrade = scheduler->approximateQueueFillGrade();
          if (fillGrade >= unavailabilityFillGrade) {
            // oops, queue is relatively full
            available = false;
          }   
        }   

While this code works if the unavailability fill grade is 0, it will lead to more requests being sent to the instance now even if the queue is relatively full.
If this is an intentional change, fine. If not, is it much work to implement the fill grade methods in the new scheduler, or is there a reason why we shouldn't do this?

[mpoeter]

These are functions defined in the Scheduler interface, but currently only make sense for the SupervisedScheduler (leaky abstraction). We will have to discuss if/how to add this to the new scheduler.

[jsteemann]

LGTM