Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added dependency-watchdog #990

Merged
merged 1 commit into from May 13, 2019

Conversation

georgekuruvillak
Copy link
Contributor

@georgekuruvillak georgekuruvillak commented May 7, 2019

What this PR does / why we need it: When etcd-main becomes ready after a prolonged period of time, the control-plane components had gone into CrashloopBackoff`. The control-plane pods may have to wait till 300 seconds before restarting containers. This fix aims to reduce the downtime caused due to pod waiting for MaxContainerBackOff to have the container recreated by restarting the control-plane pods in CrashloopBackoff when the etcd-main service gets serviceable endpoints.

Which issue(s) this PR fixes:
Fixes #148

Special notes for your reviewer:

Release note:

The [dependency-watchdog controller](https://github.com/gardener/dependency-watchdog) is now deployed next to every control plane in the seed.
dependency-watchdog checks for the readiness of a service. If the endpoints are ready to accept requests, the dependant pods as specified in the config is restarted if found to be in `CrashLoopBackoff`.

Copy link

@amshuman-kr amshuman-kr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A typo in the in yaml/json binding. Sorry, I didn't catch it earlier.

@georgekuruvillak georgekuruvillak force-pushed the add-dependency-watchdog branch 2 times, most recently from b01ff77 to efec6b5 Compare May 7, 2019 18:14
@georgekuruvillak georgekuruvillak force-pushed the add-dependency-watchdog branch 2 times, most recently from 6fa0441 to b0b650f Compare May 8, 2019 12:57
@georgekuruvillak georgekuruvillak changed the title Added dep-controller Added dependency-watchdog May 8, 2019
rfranzke
rfranzke previously approved these changes May 13, 2019
Copy link
Member

@rfranzke rfranzke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@rfranzke
Copy link
Member

Tests need to be fixed, after that it can be merged.

@rfranzke rfranzke merged commit 04ae532 into gardener:master May 13, 2019
@georgekuruvillak georgekuruvillak deleted the add-dependency-watchdog branch July 1, 2019 07:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Abort Control Plane Component CrashLoopBackOff when ETCD Returns From Maintenance
5 participants