Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update and improve ReplicationController resource lifecycle test #90880

Conversation

Riaankl
Copy link
Contributor

@Riaankl Riaankl commented May 8, 2020

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
Updates the test to rely on responses from Patch or Update instead of later Get or List

Which issue(s) this PR fixes:
Fixes: #90881
Resolves #90957

#88588 (comment)

Special notes for your reviewer:
Attempts to fix flakiness.

Does this PR introduce a user-facing change?:

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

NONE

/sig testing

@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. sig/testing Categorizes an issue or PR as relevant to SIG Testing. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels May 8, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @Riaankl. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. area/test sig/apps Categorizes an issue or PR as relevant to SIG Apps. labels May 8, 2020
@k8s-ci-robot k8s-ci-robot requested review from foxish and kow3ns May 8, 2020 03:22
@Riaankl
Copy link
Contributor Author

Riaankl commented May 8, 2020

/area conformance
/assign @johnbelamaric
/assign @spiffxp

@k8s-ci-robot k8s-ci-robot added the area/conformance Issues or PRs related to kubernetes conformance tests label May 8, 2020
@Riaankl
Copy link
Contributor Author

Riaankl commented May 8, 2020

This PR replaces #89746

@hh hh added this to In Progress Issues in conformance-definition May 8, 2020
@hh hh moved this from In Progress Issues to In Progress PRs in conformance-definition May 8, 2020
@fejta-bot
Copy link

Unknown CLA label state. Rechecking for CLA labels.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/check-cla

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 8, 2020
@hh
Copy link
Member

hh commented May 12, 2020

/retest pull-kubernetes-files-remake

@k8s-ci-robot
Copy link
Contributor

@hh: The /retest command does not accept any targets.
The following commands are available to trigger jobs:

  • /test pull-kubernetes-bazel-build
  • /test pull-kubernetes-bazel-test
  • /test pull-kubernetes-conformance-image-test
  • /test pull-kubernetes-conformance-kind-ipv6-parallel
  • /test pull-kubernetes-dependencies
  • /test pull-kubernetes-dependencies-canary
  • /test pull-kubernetes-e2e-aws-eks-1-13-correctness
  • /test pull-kubernetes-files-remake
  • /test pull-kubernetes-e2e-gce
  • /test pull-kubernetes-e2e-gce-canary
  • /test pull-kubernetes-e2e-gce-ubuntu
  • /test pull-kubernetes-e2e-gce-ubuntu-containerd
  • /test pull-kubernetes-e2e-gce-rbe
  • /test pull-kubernetes-e2e-gce-alpha-features
  • /test pull-kubernetes-e2e-gce-device-plugin-gpu
  • /test pull-kubernetes-integration
  • /test pull-kubernetes-cross
  • /test pull-kubernetes-e2e-kind
  • /test pull-kubernetes-e2e-kind-canary
  • /test pull-kubernetes-e2e-kind-ipv6
  • /test pull-kubernetes-e2e-kind-ipv6-canary
  • /test pull-kubernetes-conformance-kind-ga-only
  • /test pull-kubernetes-conformance-kind-ga-only-parallel
  • /test pull-kubernetes-e2e-kops-aws
  • /test pull-kubernetes-bazel-build-canary
  • /test pull-kubernetes-bazel-test-canary
  • /test pull-kubernetes-bazel-test-integration-canary
  • /test pull-kubernetes-local-e2e
  • /test pull-publishing-bot-validate
  • /test pull-kubernetes-e2e-gce-network-proxy-http-connect
  • /test pull-kubernetes-e2e-gce-network-proxy-grpc
  • /test pull-kubernetes-e2e-gci-gce-autoscaling
  • /test pull-kubernetes-e2e-aks-engine-azure
  • /test pull-kubernetes-e2e-azure-disk
  • /test pull-kubernetes-e2e-azure-disk-vmss
  • /test pull-kubernetes-e2e-azure-file
  • /test pull-kubernetes-e2e-gci-gce-ipvs
  • /test pull-kubernetes-node-e2e
  • /test pull-kubernetes-e2e-containerd-gce
  • /test pull-kubernetes-node-e2e-containerd
  • /test pull-kubernetes-node-e2e-alpha
  • /test pull-kubernetes-node-kubelet-serial-cpu-manager
  • /test pull-kubernetes-node-kubelet-serial-topology-manager
  • /test pull-kubernetes-e2e-gce-100-performance
  • /test pull-kubernetes-e2e-gce-big-performance
  • /test pull-kubernetes-e2e-gce-large-performance
  • /test pull-kubernetes-kubemark-e2e-gce-big
  • /test pull-kubernetes-kubemark-e2e-gce-scale
  • /test pull-kubernetes-e2e-gce-storage-slow
  • /test pull-kubernetes-e2e-gce-storage-snapshot
  • /test pull-kubernetes-e2e-gce-storage-slow-rbe
  • /test pull-kubernetes-e2e-gce-csi-serial
  • /test pull-kubernetes-e2e-gce-iscsi
  • /test pull-kubernetes-e2e-gce-iscsi-serial
  • /test pull-kubernetes-e2e-gce-storage-disruptive
  • /test pull-kubernetes-e2e-aks-engine-azure-windows
  • /test pull-kubernetes-e2e-azure-disk-windows
  • /test pull-kubernetes-e2e-azure-file-windows
  • /test pull-kubernetes-typecheck
  • /test pull-kubernetes-verify
  • /test pull-kubernetes-e2e-windows-gce

Use /test all to run all jobs.

In response to this:

/retest pull-kubernetes-files-remake

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@hh
Copy link
Member

hh commented May 12, 2020

/test pull-kubernetes-files-remake

Copy link
Member

@spiffxp spiffxp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ok-to-test
/release-note-none

It's been a while since I've looked at this, and I've got to admit the test is now so long I'm having a tough time keeping track of what is being done when and why. I'm not sure why all of the watchEvent checks have been added. Do we know they help?

Comment on lines 144 to 151
eventFound := false
for watchEvent := range rcWatchChan {
if watchEvent.Type == watch.Added {
eventFound = true
break
}
}
framework.ExpectEqual(eventFound, true, "failed to find RC %v event", watch.Added)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: for the number of times this is copy-pasted around, I would pull this into a function

expectWatchEventType(rcWatch, watch.Added)

https://github.com/kubernetes/kubernetes/blob/master/test/e2e/apimachinery/watch.go#L397-L431 has something to crib from

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using framework.WatchUntilWithoutRetry (introduced in #91416) for these checks as of 313f0341ca3cc3de91c664e6f5d001dab1a6852e

Comment on lines 209 to 219
eventFound = false
for watchEvent := range rcWatchChan {
rc, ok := watchEvent.Object.(*v1.ReplicationController)
framework.ExpectEqual(ok, true, "Unable to convert type of ReplicationController watch watchEvent")

if rc.Status.Replicas == testRcInitialReplicaCount {
eventFound = true
break
}
}
framework.ExpectEqual(eventFound, true, "RC does not have initial replica count")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this check / what is this trying to test?

If this is trying to wait until the RC status has been updated to undo what we just patched in, it should be looking at rc.Status.ReadyReplicas

Comment on lines 241 to 250
ginkgo.By("waiting for RC to be modified")

eventFound = false
for watchEvent := range rcWatchChan {
if watchEvent.Type == watch.Modified {
eventFound = true
break
}
}
framework.ExpectEqual(eventFound, true, "failed to find RC %v event", watch.Modified)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? It seems like the below watch event loop is going to wait until the RC status converges to the new number of replicas, I'm not sure why we need to wait before we wait

@@ -245,9 +301,7 @@ var _ = SIGDescribe("ReplicationController", func() {
if rcItem.ObjectMeta.Name == testRcName &&
rcItem.ObjectMeta.Namespace == testRcNamespace &&
rcItem.ObjectMeta.Labels["test-rc-static"] == "true" &&
rcItem.ObjectMeta.Labels["test-rc"] == "patched" &&
rcItem.Status.Replicas == testRcMaxReplicaCount &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be checking Spec.Replicas?

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 16, 2020
@Riaankl Riaankl moved this from PRs Needing Review to PRs Needing Approval in conformance-definition Jun 11, 2020
@Riaankl Riaankl moved this from PRs Needing Approval to Done in conformance-definition Jun 11, 2020
@Riaankl Riaankl moved this from Done to PRs Needing Approval in conformance-definition Jun 11, 2020
@Riaankl Riaankl moved this from PRs Needing Approval to PRs Needing Review in conformance-definition Jun 11, 2020
@BobyMCbobs BobyMCbobs force-pushed the ii-update-and-improve-replicationcontroller-resource-lifecycle-test branch from 9caf289 to e3b9d78 Compare June 11, 2020 02:08
@BobyMCbobs
Copy link
Member

/retest

@spiffxp
Copy link
Member

spiffxp commented Jun 11, 2020

/test pull-kubernetes-verify

@spiffxp
Copy link
Member

spiffxp commented Jun 16, 2020

/release-note-none

/approve
/lgtm
to get us some signal, I'm still not convinced this is the right way to go, will summarize in another comment

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 16, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Riaankl, spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 16, 2020
@BobyMCbobs
Copy link
Member

/retest

1 similar comment
@BobyMCbobs
Copy link
Member

/retest

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@Riaankl
Copy link
Contributor Author

Riaankl commented Jun 17, 2020

/test pull-kubernetes-e2e-kind

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit cf36d9b into kubernetes:master Jun 18, 2020
conformance-definition automation moved this from PRs Needing Review to Done Jun 18, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.19 milestone Jun 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/conformance Issues or PRs related to kubernetes conformance tests area/e2e-test-framework Issues or PRs related to refactoring the kubernetes e2e test framework area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
Development

Successfully merging this pull request may close these issues.

Watch event test verification tooling Write CoreV1ReplicationController lifecycle test+promote - +7 coverage
7 participants