Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graceful shutdown for Kubernetes #35

Closed
alexellis opened this issue Feb 6, 2020 · 0 comments · Fixed by #36
Closed

Graceful shutdown for Kubernetes #35

alexellis opened this issue Feb 6, 2020 · 0 comments · Fixed by #36

Comments

@alexellis
Copy link
Member

There appears to be an issue with graceful shutdowns. This occurs when Kubernetes sends a SIGTERM to our function's Pod.

We have two processes:

  • of-watchdog
  • Go handler / HTTP server

The of-watchdog catches the signal and waits for wait_timeout env-var, but the child process appears to die immediately

We need to copy the code / approach of the of-watchdog into the Go handler for all templates and test.

Task

  • Show the issue by adding a SIGTERM handler and echo message in the entry point for one of the templates
  • Deploy in a container
  • Scale to --replicas=0
  • Look at the logs

Now fix the code and repeat

You should see that both processes wait at least wait_timeout before exiting.

alexellis added a commit that referenced this issue Feb 6, 2020
Closes: #35

Tested outside of a container, using kill -s TERM <PID> - it
worked as expected by waiting for N seconds.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
alexellis added a commit that referenced this issue Feb 6, 2020
Closes: #35

Tested outside of a container, using kill -s TERM <PID> - it
worked as expected by waiting for N seconds.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant