Improve fairness between system selector and custom readiness queue

[carllerche]

The current implementation of Poll::poll always polls the system selector first, followed by the custom readiness queue. This means that large number of inbound system events could starve polling the custom readiness queue.

See: #537 (comment)

[frehberg]

As far as I understand the current implemenation, a thread may wait for N events and grab them all. So that in a multi-threaded environment a single thread may end up with N events to be processed, but other threads might be idling, waiting for more events to show up.

Frameworks such as boost::asio and TAO/ACE implement a so called Reactor-Frameworks, that scales well on multi-threaded/multi-ore systems. The implementation is similar to current one, but with the difference that any thread gets only the first event from ReadynessQueue (here the joined queue of system-events and custom-events)

If the ReadynessQueue is empty, a thread grabs the lock and performs a poll for MAX events (system and custom). On success, assuming N events occured, the (N-1) events are stored in the ReadynessQueue and the active thread returns with single event to be processed. All other blocked threads are woken up and may grab the first pending event from ReadynessQueue.

This way the load would be balanced well on multiple threads, instead of single thread grabbing all events at once. System events and custom events would be handled fair.

AFAICS, it would not be a big modification to current code base, just the libraries on top of mio would see a different behavior, (regarding number of events being N=1 always). Maybe the Poll-impl should provide a dedicated constructor, to choose the right size of the ReadynessQueue and value MAX

Update: AFAICS, it would required to introduce kind of PendingQueue, joining system-events and custom-events. It would be a linked list of pre-allocated Nodes, one for each Registration. The system-selector might be called using an internal epoll-event-array of length K, or it might be parameter of Poll:poll(..) of hte active thread succeeding to grab the epoll-lock.

Implementing this behavior, the signature of poll might be even simplified, such as:

Poll::poll(timeout) -> io::Result<Event>

Any comments are welcome.

[frehberg]

The following patch (work in progress) would balance between system events and custom events, the sleep_marker would be used to punctuate in ReadinessQueue a moment in time the last system event occured. When consuming events from ReadinessQueue and reaching this marker, it will indicate that the system-selector needs to be invoked again (using the user-defined timeout)

#772

[carllerche]

@frehberg re the original comment, I think the architecture could work for a multi threaded reactor setup, but I am not sure if this is something for Mio or for other libs. That said, from my interpretation of your description, I don't see how that sort of setup would have great thread affinity? Generally, you are going to want to process the same I/O resource on the same thread as much as possible.

Mio, at some point, will probably want to go multi threaded, but that would only be to expose OS level capabilities.

[carllerche]

This feature will (most likely) be deprecated: #907. As such, this issue is no longer relevant.