core: improve graceful shutdown

[dormando]

WIP code for making graceful shutdown more useful. Goals:

• support SO_REUSEPORT (optionally) so multiple daemons can listen at the same time.
• Stop listener sockets to start the shutdown process.
• Allow in-flight requests to complete.
• Optionally forecfully close connections when they return to an idle state.

Existing code quickly closes all connections, regardless of if they are in the middle of handling user requests.

STATUS:

• Usable for basic QA testing.

TODO:

• configure.ac test for SO_REUSEPORT
• start argument for whether to use SO_REUSEPORT
• account for and wait on backgrounded proxy requests to complete (async)
• stop proxy threads from the thread stop code (IO, config, object manager)
• option to wait for client connections to naturally close (or some kind of time deadline)
• code audit to ensure usage of c->thread is appropriate during graceful shutdown
• unit tests to try shutdown while proxy client connections are in various states.

[dormando]

The two things I was most worried about (tracking proxy in-flight + c->thread being safe) turned out to be super easy.

Unfortunately I noticed the proxy in-flight request counters are on the threadlocal stats structure, which is resettable... so things would break if someone ran stats reset. I'll have to fix that separately.

stats_state is usually used for stats counters that shouldn't be reset, but they are centralized and globally locked which I was trying to avoid here.