Ensure concierge and supervisor gracefully exit

[enj]

Changes made to both components:

1. Logs are always flushed on process exit
2. Informer cache sync can no longer hang process start up forever

Changes made to concierge:

1. Add pre-shutdown hook that waits for controllers to exit cleanly
2. Informer caches are synced in post-start hook

Changes made to supervisor:

1. Add shutdown code that waits for controllers to exit cleanly
2. Add shutdown code that waits for active connections to become idle

Waiting for controllers to exit cleanly is critical as this allows
the leader election logic to release the lock on exit. This reduces
the time needed for the next leader to be elected.

Signed-off-by: Monis Khan mok@vmware.com

Fixes #654
xref #828

Release note:

The pinniped supervisor now waits for up to a minute for active connections to become idle before exiting when told to terminate by the kubelet.

[codecov]

Codecov Report

Merging #829 (0d285ce) into main (e43bd59) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #829      +/-   ##
==========================================
- Coverage   78.66%   78.64%   -0.02%     
==========================================
  Files         129      129              
  Lines        8941     8945       +4     
==========================================
+ Hits         7033     7035       +2     
- Misses       1680     1682       +2     
  Partials      228      228              

Impacted Files	Coverage Δ
internal/concierge/server/server.go	27.50% <0.00%> (-0.47%)	⬇️
internal/leaderelection/leaderelection.go	52.12% <0.00%> (-1.14%)	⬇️
...l/localuserauthenticator/localuserauthenticator.go	57.40% <0.00%> (+0.92%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e43bd59...0d285ce. Read the comment docs.