Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix preemption tests that use PDB #62933

Merged
merged 2 commits into from May 23, 2018

Conversation

@bsalamat
Copy link
Contributor

bsalamat commented Apr 21, 2018

What this PR does / why we need it:
Scheduler integration tests that test preemption in presence of PDB had an issue causing PDB status not getting updated. This PR fixes the issue.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Release note:

NONE

/sig scheduling

xref/ #57057

// Create PDBs.
for _, pdb := range test.pdbs {
_, err := context.clientSet.PolicyV1beta1().PodDisruptionBudgets(context.ns.Name).Create(pdb)
if err != nil {
t.Fatalf("Failed to create PDB: %v", err)
}
}
// Wait for PDBs to be stable
for i, pdb := range test.pdbs {
waitPDBStable(t, context.clientSet, test.pdbPodNum[i], context.ns.Name, pdb.Name)

This comment has been minimized.

@k82cn

k82cn Apr 21, 2018

Member

Should we check pdb stable in scheduler cache directly?

This comment has been minimized.

@bsalamat

bsalamat Apr 21, 2018

Author Contributor

Good point. I think that is a safer approach to ensure that PDB is updated when scheduler processes it. I will make the change.

@k82cn

This comment has been minimized.

Copy link
Member

k82cn commented Apr 21, 2018

LGTM :)

@resouer
Copy link
Member

resouer left a comment

the main test logic lgtm except for bazel failure :)

@k8s-ci-robot k8s-ci-robot added size/L and removed size/M labels Apr 23, 2018

@bsalamat

This comment has been minimized.

Copy link
Contributor Author

bsalamat commented Apr 23, 2018

/hold

I am still working on this

@greghaynes
Copy link
Contributor

greghaynes left a comment

I verified this fixes the issue I was encountering in #57057. Just a couple of minor comments

},
}

for _, test := range tests {
fmt.Printf("==================BOBBY: running test %v\n", test.description)

This comment has been minimized.

@greghaynes

greghaynes Apr 24, 2018

Contributor

Did you mean to remove this?

This comment has been minimized.

@bsalamat

bsalamat Apr 24, 2018

Author Contributor

Yes. These are part of the "experimenting" commit and that's why I have put a hold on this PR. I will remove them when the PR is ready.
I am seeing scheduler's cache getting out of sync. I have noticed that it happens only when tests are created with a separate "pod informer" (in initTestSchedulerWithOptions). I would like to find the root-cause before merging this PR.

This comment has been minimized.

@greghaynes

greghaynes Apr 24, 2018

Contributor

Ah, sorry, I missed the hold :)

// Wait for PDBs to show up in the scheduler's cache.
if err := wait.Poll(time.Second, 15*time.Second, func() (bool, error) {
//// Wait for PDBs to be stable
//for i, pdb := range test.pdbs {

This comment has been minimized.

@greghaynes

greghaynes Apr 24, 2018

Contributor

Ditto

@bsalamat bsalamat force-pushed the bsalamat:fix_pdb branch 2 times, most recently from 11ab423 to 9ee09d3 Apr 27, 2018

@bsalamat

This comment has been minimized.

Copy link
Contributor Author

bsalamat commented Apr 27, 2018

The failed integration test is caused by #63262. This should pass after #63264 is merged.

@bsalamat bsalamat force-pushed the bsalamat:fix_pdb branch from 9ee09d3 to 422b324 May 17, 2018

@bsalamat

This comment has been minimized.

Copy link
Contributor Author

bsalamat commented May 17, 2018

/hold cancel

@bsalamat

This comment has been minimized.

Copy link
Contributor Author

bsalamat commented May 17, 2018

I rebased this. It should be good to go now.

@bsalamat bsalamat force-pushed the bsalamat:fix_pdb branch from 422b324 to a0b0c15 May 17, 2018

@bsalamat

This comment has been minimized.

Copy link
Contributor Author

bsalamat commented May 17, 2018

/retest

@bsalamat bsalamat added this to the v1.11 milestone May 21, 2018

@bsalamat

This comment has been minimized.

Copy link
Contributor Author

bsalamat commented May 22, 2018

/retest

@k82cn

This comment has been minimized.

Copy link
Member

k82cn commented May 22, 2018

/lgtm

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented May 22, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bsalamat, k82cn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-github-robot

This comment has been minimized.

Copy link
Contributor

k8s-github-robot commented May 22, 2018

[MILESTONENOTIFIER] Milestone Pull Request: Up-to-date for process

@bsalamat @k82cn

Pull Request Labels
  • sig/scheduling: Pull Request will be escalated to these SIGs if needed.
  • priority/important-soon: Escalate to the pull request owners and SIG owner; move out of milestone after several unsuccessful escalation attempts.
  • kind/bug: Fixes a bug discovered during the current release.
Help
@k82cn

This comment has been minimized.

Copy link
Member

k82cn commented May 22, 2018

/retest

@fejta-bot

This comment has been minimized.

Copy link

fejta-bot commented May 22, 2018

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel comment for consistent failures.

@k8s-github-robot

This comment has been minimized.

Copy link
Contributor

k8s-github-robot commented May 23, 2018

Automatic merge from submit-queue (batch tested with PRs 63914, 63887, 64116, 64026, 62933). If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit b5cd7d8 into kubernetes:master May 23, 2018

18 checks passed

Submit Queue Queued to run github e2e tests a second time.
Details
cla/linuxfoundation bsalamat authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.