Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alter the logic of "updated" pods in WaitForControlledPodsRunning measurement #1547

Open
tosi3k opened this issue Oct 27, 2020 · 3 comments
Open
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness.

Comments

@tosi3k
Copy link
Member

tosi3k commented Oct 27, 2020

Before #1035 was merged, WaitForControlledPodsRunning barrier was simply waiting for all pods specified by a label selector to reach the running state. Such approach doesn't take into account potential rolling updates of pods belonging to certain deployment(s). #1035 introduced an additional check that the appropriate pods' specs match the deployment's template spec as a heuristic to ensure that we're finished with the rolling updates.

This approach doesn't work when there's an additional server logic altering the pods' specs, however. There are two possible ideas how to improve this:

  1. User explicitly says which field should be used to distinguish old and new version, e.g. if rolling update changes some env variable or image, we can add a new WaitForControllerPodsRunning's parameter next to checkIfPodsAreUpdated like "fieldToCheck: 'spec.containers[0].env[0]'.
  2. Implementing more like three way diff:
    • If the given deployment is being created for the first time, all pods inside are "updated"
    • If we do some update to deployment's spec.template.spec, we do "diff" of previous spec.template.spec with the updated state spec.template.spec (with intent to automatically find modified field) and then check if this "diff" matches a pod's spec.

Option 2 seems to be easier to use, but I'm not sure if we can implement this without larger CL's refactoring (e.g. do we store old version of each deployment)?

Option 1 seems to be easier to implement and is more explicit, but requires some manual changes for all WaitForControlledPodsRunning that wants to use "pod updated" feature.

@tosi3k tosi3k added the kind/bug Categorizes issue or PR as related to a bug. label Oct 27, 2020
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 25, 2021
@tosi3k
Copy link
Member Author

tosi3k commented Jan 25, 2021

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 25, 2021
@tosi3k
Copy link
Member Author

tosi3k commented Mar 13, 2021

/lifecycle frozen

@k8s-ci-robot k8s-ci-robot added the lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. label Mar 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness.
Projects
None yet
Development

No branches or pull requests

3 participants