Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix conflicted cache when the requests are canceled by other Azure operations #81282

Merged
merged 1 commit into from Aug 12, 2019

Conversation

@feiskyer
Copy link
Member

commented Aug 12, 2019

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

/kind bug

What this PR does / why we need it:

When a new requests are sending to an Azure resource, the ongoing requests may be canceled with error:

Code="Canceled" Message="Operation was canceled." Details=[{"code":"CanceledAndSupersededDueToAnotherOperation","message":"Operation PutLoadBalancerOperation (c29592fc-da61-47c3-b032-b4a2df0cc62a) was canceled and superseded by operation PutLoadBalancerOperation (b0a47269-40c5-461a-b704-2f162f8300f3)."}]

In such case, the resource has already been changed by other requests. So this PR invalidates the cache and then abort the backoff retry. In next controller loop run, cloud provider would GET a new resource from Azure API, and then try updating again.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer: Need cherry pick to v1.15.

Does this PR introduce a user-facing change?:

Fix conflicted cache when the requests are canceled by other Azure operations.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


/kind bug
/area provider/azure
/priority critical-urgent
/sig cloud-provider

@feiskyer feiskyer force-pushed the feiskyer:fix-az-cancel branch from e6ac7af to f692944 Aug 12, 2019
@feiskyer

This comment has been minimized.

Copy link
Member Author

commented Aug 12, 2019

/assign @andyzhangx

@feiskyer

This comment has been minimized.

Copy link
Member Author

commented Aug 12, 2019

/test pull-kubernetes-e2e-aks-engine-azure

Copy link
Member

left a comment

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 12, 2019
@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Aug 12, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx, feiskyer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fejta-bot

This comment has been minimized.

Copy link

commented Aug 12, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit ae45744 into kubernetes:master Aug 12, 2019
24 checks passed
24 checks passed
cla/linuxfoundation feiskyer authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-conformance-image-test Skipped.
pull-kubernetes-cross Skipped.
pull-kubernetes-dependencies Job succeeded.
Details
pull-kubernetes-e2e-aks-engine-azure Job succeeded.
Details
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-csi-serial Skipped.
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gce-iscsi Skipped.
pull-kubernetes-e2e-gce-iscsi-serial Skipped.
pull-kubernetes-e2e-gce-storage-slow Skipped.
pull-kubernetes-godeps Skipped.
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped.
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-node-e2e-containerd Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped.
tide In merge pool.
Details
Provider Azure automation moved this from In progress to Done Aug 12, 2019
@k8s-ci-robot k8s-ci-robot added this to the v1.16 milestone Aug 12, 2019
@feiskyer feiskyer deleted the feiskyer:fix-az-cancel branch Aug 13, 2019
k8s-ci-robot added a commit that referenced this pull request Aug 14, 2019
…82-upstream-release-1.15

Automated cherry pick of #81282: Fix conflicted cache when the requests are canceled by other Azure requests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.