Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase image shutdown timeout for deployment #159

Open
2 tasks
Noah-Huppert opened this issue Jul 11, 2019 · 0 comments
Open
2 tasks

Increase image shutdown timeout for deployment #159

Noah-Huppert opened this issue Jul 11, 2019 · 0 comments
Labels
enhancement New feature or request

Comments

@Noah-Huppert
Copy link
Contributor

The server will not shut down until all jobs have finished running.

If the server does not shut down gracefully in some timeout k8s will kill 9 the pod. This will screw up any jobs which are running.

  • Increase the timeout to something pretty big, maybe 2 minutes, so that this never happens.
  • Look into k8s pods in the terminating state to make sure that no new connections are sent to these pods
    • If new connections are sent to terminating pods then simply setting the timeout to be higher will not be enough. As the server could receive a web hook request and start a new job, which in turn would make the server wait even longer before shutting off, one could imagine a situation where the terminating pod keeps receiving web hook requests and never gracefully shuts down, causing a kill 9
@Noah-Huppert Noah-Huppert added the enhancement New feature or request label Jul 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant