Do not send redundant SIGQUITs

[brsakai-csco]

If the worker reports itself as shutting down, do not second another SIGQUIT to the worker. It already knows it is shutting down, and this can cause a race with signal handlers that triggers coredumps.

Summary

Full writeup in #2046
The short version:

• When a worker receives a SIGQUIT, it begins to gracefully shut down
• During graceful shutdown, the worker notifies the manager thread of its shutdown
• If the manager sent the initial SIGQUIT, then it already knows the worker is shutting down and takes no action
• If the manager did not send the initial SIGQUIT, then it marks the worker for graceful shutdown, causing it to send a second SIGQUIT
• This second SIGQUIT can arrive once Perl has already de-registered the QUIT signal handler, which causes it to perform the default QUIT action (coredump)

The patch here increments the worker's quit state in the managing thread so that it does not send a second SIGQUIT to the worker if the manager finds out about the graceful shutdown from the worker

Motivation

This behavior was causing coredumps in our application, since we were using SIGQUIT to gracefully end workers.
I understand from #1883/#1449 that there are also other ways for the worker to initiate graceful shutdown, which can trigger the same race condition

References

Tested via the test app uploaded to #2046
Attempts to fix the same issue seen in #1883

[marcusramberg]

I like this fix better, except for the if duplication/perltidy.

[kraih]

Please squash your commits.

[brsakai-csco]

@kraih Done, I think. Let me know if that doesn't look how you expect

[kraih]

Is do not second another SIGQUIT to the worker (from the commit message) correct english?

[brsakai-csco]

I didn't even see that. Must've switched the words in my head without realizing. Something like do not send a second > do not send another > do not second another.

Updated the commit message 👍

[jixam]

I like this 👍

For completeness, just a note that this does not remove the race condition. It is still possible to hit it by sending two SIGQUIT signals close to each other or by sending a single signal when a worker is already shutting down.

To fully fix the race, $SIG{QUIT} = 'IGNORE'; is needed before exit 0; in the worker. However, this is not trivial to do because the handler is localized.