Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tighten maximum retry loop for aggregate api availability #68678

Merged
merged 1 commit into from
Oct 1, 2018
Merged

tighten maximum retry loop for aggregate api availability #68678

merged 1 commit into from
Oct 1, 2018

Conversation

deads2k
Copy link
Contributor

@deads2k deads2k commented Sep 14, 2018 •
edited
Loading

The aggregated API availability controller listens to the API, but because it relies on the routability of the
service or pod network, it is possible for an external, non-watchable factor to affect availability. This pull keeps the maximum disruption time to a minimum, but it still prevents hot loops.

@kubernetes/sig-api-machinery-pr-reviews
@sttts @mfojtik this should help responsiveness

apiservice availability related to networking glitches are corrected faster

@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Sep 14, 2018
deads2k
deads2k commented Sep 14, 2018
View reviewed changes
staging/src/k8s.io/kube-aggregator/pkg/controllers/status/available_controller.go
// We want a fairly tight requeue time. The controller listens to the API, but because it relies on the routability of the
// service network, it is possible for an external, non-watchable factor to affect availability. This keeps
// the maximum disruption time to a minimum, but it does prevent hot loops.
workqueue.NewItemExponentialFailureRateLimiter(5*time.Millisecond, 30*time.Second),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the default that is used for most controllers has a max retry time of 1000*seconds instead of 30 seconds like I've done here.

@deads2k
Copy link
Contributor Author

deads2k commented Sep 14, 2018

/kind bug

This results in a longer than reasonable outage during blips.

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. and removed needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Sep 14, 2018
@sttts
Copy link
Contributor

sttts commented Sep 17, 2018

Sounds reasonable. Giving some other eyes a chance to look at it.

@mfojtik
Copy link
Contributor

mfojtik commented Sep 17, 2018

👍

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Sep 17, 2018
tnozicka
tnozicka reviewed Sep 17, 2018
View reviewed changes
Copy link
Contributor

@tnozicka tnozicka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@wenjiaswe
Copy link
Contributor

/assign @cheftako

cheftako
cheftako approved these changes Oct 1, 2018
View reviewed changes
Copy link
Member

@cheftako cheftako left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 1, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cheftako, deads2k, tnozicka

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit 2a3c8d5 into kubernetes:master Oct 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cheftako cheftako cheftako approved these changes

@tnozicka tnozicka tnozicka approved these changes

@lavalamp lavalamp Awaiting requested review from lavalamp

@smarterclayton smarterclayton Awaiting requested review from smarterclayton

Assignees

@cheftako cheftako

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
8 participants
@deads2k @sttts @mfojtik @wenjiaswe @k8s-ci-robot @fejta-bot @cheftako @tnozicka