Implement an algorithm to make this crate no_std, take 3

[notgull]

This PR supersedes #33 and aims to improve on its strategy using a hybrid between its strategy and the strategy that's currently used. Basically, it keeps the master strategy, but replaces the Mutex with another implementation based on the event queue in #33.

To clarify, the current strategy used on the master branch is to use the std::sync::Mutex type to wrap a !Sync linked list built using the Cell type. At its core, event-listener is just a queue of wakers that we wake up whenever we notify the event. On modern operating systems, this is very fast. Mutexes are implemented using futexes, meaning that in the case of no contention there are, at most, two atomic operations involved. One to lock the mutex, and one to unlock. The main disadvantage of this approach is that the Mutex requires libstd to be linked, which forces the library (and anything that depends on it) to not be no_std. One of my goals is to be able to use smol on embedded systems without needed to resort to embassy, an RTOS or something similar. While async-io, blocking and their dependents are not usable on these platforms, it would be nice to at least have an async toolkit for mutexes, et al.

My strategy for #30 and #33 was to reimplement event-listener by replacing this list with a fully atomic list. In #30, I used the concurrent-queue crate to implement the listener queue, and in #33 I used a homegrown singly atomic linked list implementation. The main advantage of these implementations is that they are completely lock-free. Although I didn't test for this, I imagine that in both of these implementations, worst-case contention would be reduced significantly. In addition, since they only used atomics and not locks, they were available on no_std platforms. In fact, by taking advantage of portable-atomic, it could even run on platforms without atomic support.

The primary disadvantage of these approaches was speed. #30 exhibited a slowdown of up to 50%, which was completely unacceptable. Even through certain optimizations, I couldn't get it to break 30%. #33 did better; at best, it only exhibited a 15% slowdown. Still, it's imposing a 15% performance tax on all users, including the admittedly larger pool of use cases that would only ever use libstd. The ideal solution would be to not incur any performance issues over the current system.

A theoretically ideal solution would be to replace the Mutex with a spinlock. It's unlikely to occur much contention anyways, so in the ideal case (and, arguably, the overwhelming majority of usecases) it would be identical to the current system while also being no_std. However, spinlocks are dangerous, and a non-starter anyways.

This solution began with the idea that we could use this theoretical spinlock in the best case and then fall back to the listener queue case when there's too much contention. This "hybrid" approach would allow us to utilize the performance gains of the spinlock while also being able to fall back onto the queue for the worst cases. While this approach is difficult to directly implement in Rust, I've created a modified version of this approach that should be able to emulate it well enough.

This new implementation is based on the idea that, in the ideal use case, the inner Mutex should never be contended. The Mutex in the Event structure is replaced with an optimistic spinlock. In the best case, the crate functions much like it does today It will try to lock the mutex, and if it does, performs the desired operation (e.g. inserting into the listener list, notifying an entry, etc.). However, if the mutex is locked, it pushes a node representing the desired operation onto the backup queue. The operations that can be performed are as follows:

• Push a listener into the list.
• Notify listeners.
• Remove a listener from the list, and optionally propagate its notification if it has one.
• Wake a waker. There are some operations that I cannot effectively emulate with this system, so the waker just waits for the mutex to be available.

Once whoever holds the mutex lock is about to unlock it, they check this backup queue. If there are any operations, they are popped off of the queue and ran.

There is definitely room for improvement in this system, but speed-wise, I've accomplished my goal. From benchmarks, it's at least on par with the master implementation, and even seems to be faster in certain cases.

Future work would involve making the spinlock fair, as well as implementing portable_atomic and loom features.

Things I need to do before marking this PR as ready to review:

• Determine semantics.
• The new event queue needs more testing.
• Clean up code overall.
• MIRI is failing, determine why. MIRI seems to be deadlocking? I'm not sure why, maybe it's a MIRI bug? With the new queue implementation this issue appears to have vanished.

[notgull]

MIRI is deadlocking for some reason? I can't tell if it's the code or if it's a MIRI bug. Anyways, I've opened this for review now.

[fogti]

Cargo hack build does not seem to support --no-dev-dep (anymore?) idk what's up with that, but it makes CI fail.

[notgull]

I just took a look, I typo'd no-dev-deps (with an s at the end), let me fix that.

[fogti]

Suggestion: 9c31ae6 (refactors Node such that the Drop handler is moved to another struct, such that the Node itself can be freely deconstructed, and moves the dealing with the Option to the place where it actually matters)

[notgull]

Suggestion: 9c31ae6 (refactors Node such that the Drop handler is moved to another struct, such that the Node itself can be freely deconstructed, and moves the dealing with the Option to the place where it actually matters)

Great idea! I've merged your commit into this branch.

[fogti]

Maybe in modules which aren't pub-reachable from the top module, we should replace pub(crate) with pub (which are equivalent in that case), reducing clutter.

[notgull]

Maybe in modules which aren't pub-reachable from the top module, we should replace pub(crate) with pub (which are equivalent in that case), reducing clutter.

I generally like to make it pub(crate) even if it's not reachable from the outside, for two reasons:

• Makes it slightly harder to have API leaks.
• I usually program with warn(clippy::pedantic), which issues a warning if it sees a pub where a pub(crate) would work.

[fogti]

Can we get current benchmark stats?

[notgull]

Can we get current benchmark stats?

I took some benchmarks and found that they had regressed.

After some flamegraphing, I found that this regression was because of the introduction of concurrent-queue into the hot path of dropping the ListGuard. I guess the old version is faster because checking if the queue is empty is a single atomic operation, which may be able to be more accurately predicted-on?

Anyways, I restored the old Queue implementation, and performance returned to being roughly on-par with the current implementation.

[notgull]

I also fixed the MIRI deadlock... does MIRI not support compare_exchange_weak? Weird.

[fogti]

As the problem with the previous iterations was a bigger regression, and the current benchmarks indicate that that is no longer the case (the new branch and old branch have comparable iteration time, and the noise is relatively strong in both). Can we squash-merge this?

[fogti]

Something I'd like to do next is bumping criterion, as this massively reduces the amount of pulled-in crates on dev-builds.

[notgull]

Can we squash-merge this?

Sounds good! Do you mind approving the PR so I can merge it?