Use a proper Condvar for blocking the producer

[HadrienG2]

Unlike TripleBuffer, SPMCBuffer is not always wait-free for the producer. There are situations where the producer will block for an unbounded amount of time, the simplest scenario being when the producer attempts a buffer swap and all other buffers are currently held by consumers who do not want to move to the latest buffer version yet.

We currently handle this by with a spin loop. But that can be a waste of CPU time. Instead, we should use the OS facilities for blocking the thread. Rust provides two different accesses to them, one being thread::park/unpark and the other being Condvars. Since the identity of the producer thread can change over time, I think it's best to use a Condvar.

There is a catch, however. When the producer looks for spare buffers, it does not lock a mutex, and allows consumers to liberate buffers concurrently. That's a feature, because SPMCBuffer is designed to be efficient in the wait-free regime, where producer blocking is rare, and having consumers lock a mutex on every read buffer switch comes with a scalability tax. But it means that at the time where the producer does decide to go to sleep due to storage exhaustion, storage may actually already have been concurrently liberated by consumers, which means that a producer should not solely count on consumers to wake it up on storage liberation, but also check again from time to time.

---

Here is how I propose to handle this. On the data structure side, I would add the following:

• A private mutex in the producer interface
• A shared Condvar, used for producer sleep/wakeup
• A shared AtomicBool, used to signal when the producer is going to sleep, initially false

The producer's slow "no buffer found" path would be modified as follows:

• Lock the private mutex (we don't need it, but pthread insists on it ^^)
• Set the shared flag, notifying consumers that we are going to sleep
• Block on the condvar, with a certain initial timeout T0
• Once the wait is over, clear the shared flag, unlock the mutex, and try again
• As long as we don't find a spare buffer, increase the timeout, up to a certain limit T1, by a multiplicative factor M > 1 (like exponential back-off). When we find a buffer, reduce or reset the timeout.

The consumer's buffer liberation path would be extended with the following procedure:

• Check if the shared "producer going to sleep" flag is set (keeping the fast path as fast as possible)
• If so, notify the producer that a buffer has been liberated (by calling notify_one() on the condvar)

It would be tempting to also unset the flag along the way. But wakeups may be lost since there is a delay between the moment where a producer sets the "going to sleep" flag and actually goes to sleep. As the producer will clear the flag right after waking up, I think that optimizing the optimization like this would just be an unnecessary increase of the lost wakeup risk for the sake of hypothetical CPU performance in an overall rare event.