Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Ensure connection reaper threads respawn in forks #36998
Adapted from #36979
This ensures that the connection pool reaper is respawned in forked processes. This is done by tracking the threads and verifying that they are alive.
This includes a previously failing test case that checks that connections are reaped within a fork.
Aug 20, 2019
Continuing the discussing from #36979 (comment)
@jhawthorn This is intentional because a pool is discarded after forking where the
If we don't remove the reference to the parent pool from the class instance variable during
Also, I feel that explicitly removing the reference of the pool in the reaper makes the intent of cleaning up references for the pool clearer.
@jhawthorn I think we're still missing something because running the following script on latest will show the reaper thread crashing in the forked process. The thread will only be re-spawned if a connection is re-established after it has crashed which I don't think is ideal.
Let's tackle this in another PR. I think we should probably make reap/flush return early in that case and can test that on its own (without needing the rely on the reaper) by calling those methods directly.
I might be wrong but I think from memory that unowned pools are always discarded first before new pools are initialized.
Based on the above which @matthewd can confirm, I think the intent is that a discarded pool should not be interacted with.
Also, I'm still of the view that deregistering a pool from the reaper is a much simpler solution since it only involves deleting the pool from the hash which the reaper tracks. With the weakref approach, we have to check on every loop for