Flush data from the application thread

[digitalresistor]

To speed up how soon the connected client sees data we now attempt to
flush data from the application thread when we get new data to write to
the socket.

This saves us the need to wake up the main thread, which would then
return from select(), process all sockets, look for the ones that are
writable, and then call select() again. When that select() would return
it would finally start writing data to the remote socket.

There was also no gaurantee that the main thread would get the lock for
the output buffers, and it would not be able to write any data at all
thereby looping on select() until the application thread had written
enough data to the buffers for it to hit the high water mark, or the
response was fully buffered, potentially overflowing from memory buffers
to disk.

If the socket is not ready for data, due it being non-blocking, we will
not flush any data at all, and will go notify/wake up the main thread to
start sending the data when the socket is ready.

Delivery of first byte from the WSGI application to the remote client is
now faster, and it may alleviate buffer pressure. Especially if the
remote client is connected over localhost, as is the case with a load
balancer in front of waitress.

[digitalresistor]

For reference here is some comparisons between what Waitress 2.0.0+unreleased (before this changeset, and after):

import logging

log = logging.getLogger(__name__)


def application(environ, start_response):
    start_response(
        "200 OK",
        [
            ("Content-Type", "application/octet-stream"),
        ],
    )
    for i in range(1000000):
        #log.info(f'yield {i}')
        yield f'this is iter {i}\n'.encode('utf8')


if __name__ == "__main__":
    import waitress

    logging.basicConfig(
        format="%(asctime)-15s %(levelname)-8s %(name)s %(message)s",
        level=logging.DEBUG,
    )

    waitress.serve(application)

Without these changes:

    namelookup:  0.010633s
       connect:  0.011085s
    appconnect:  0.000000s
   pretransfer:  0.011126s
      redirect:  0.000000s
 starttransfer:  11.772429s
    time_total:  17.965768s
   size_header:  161 bytes
 size_download:  19888890 bytes
speed_download:  1107043 bytes/sec
  speed_upload:  0 bytes/sec
---------------
         total:  17.965768s

WIth these changes applied:

    namelookup:  0.010248s
       connect:  0.010668s
    appconnect:  0.000000s
   pretransfer:  0.010713s
      redirect:  0.000000s
 starttransfer:  0.011079s
    time_total:  11.018954s
   size_header:  161 bytes
 size_download:  19888890 bytes
speed_download:  1804970 bytes/sec
  speed_upload:  0 bytes/sec
---------------
         total:  11.018954s