New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automated cherry pick of #67562: Prevent resourceVersion updates for custom resources on no-op writes #67616

Merged
merged 1 commit into from Aug 28, 2018

Conversation

@nikhita
Member

nikhita commented Aug 20, 2018

Cherry pick of #67562 on release-1.11.

#67562: Prevent resourceVersion updates for custom resources on no-op writes

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita
Member

nikhita commented Aug 20, 2018

@k8s-ci-robot k8s-ci-robot requested review from liggitt and removed request for david-mcmahon Aug 20, 2018

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

Currently, resourceVersion is incremented even for no-op writes for custom resources. This occurs because metadata accessors for Unstructured objects do not respect omitempty semantics for ObjectMeta fields.

Combined with ObjetMeta pruning (which respects omitempty semantics), this led to different bytes each time a custom resource was written to. Due to different bytes, resourceVersion was incremented.

Instead of updating the accessors (which could be a breaking change), we are making sure that we don't set the fields in BeforeUpdate and BeforeCreate if they have zero values. I updated the accessors in another PR against master: #67635.

More details in #67562, #67541 (comment).

/assign @foxish
for cherry-pick approval

Member

nikhita commented Aug 20, 2018

Currently, resourceVersion is incremented even for no-op writes for custom resources. This occurs because metadata accessors for Unstructured objects do not respect omitempty semantics for ObjectMeta fields.

Combined with ObjetMeta pruning (which respects omitempty semantics), this led to different bytes each time a custom resource was written to. Due to different bytes, resourceVersion was incremented.

Instead of updating the accessors (which could be a breaking change), we are making sure that we don't set the fields in BeforeUpdate and BeforeCreate if they have zero values. I updated the accessors in another PR against master: #67635.

More details in #67562, #67541 (comment).

/assign @foxish
for cherry-pick approval

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

/sig api-machinery
/kind bug
/area custom-resources

Member

nikhita commented Aug 20, 2018

/sig api-machinery
/kind bug
/area custom-resources

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

/hold
until #67562 is merged

Member

nikhita commented Aug 20, 2018

/hold
until #67562 is merged

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

Updated to reflect changes in #67562.

Member

nikhita commented Aug 20, 2018

Updated to reflect changes in #67562.

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Aug 20, 2018

Member

/lgtm

Member

liggitt commented Aug 20, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 20, 2018

@nikhita nikhita changed the title from Automated cherry pick of #67562: Prevent resource version to custom resource on no-op writes to Automated cherry pick of #67562: Prevent resourceVersion updates for custom resources on no-op writes Aug 20, 2018

Prevent resource version to custom resource on no-op writes
For ObjectMeta pruning, we round trip through marshalling and
unmarshalling. If the ObjectMeta contained any strings with "" (or other
fields with empty values) _and_ the respective fields are omitempty,
those fields will be lost in the round trip process.

This makes ObjectMeta after the no-op write different from the one
before the write.

Resource version is incremented every time data is written to etcd.
Writes to etcd short-circuit if the bytes being written are identical
to the bytes already present. So this ends up incrementing the
resourceVersion even on no-op writes.

The zero values are set in BeforeCreate and BeforeUpdate. This commit
updates BeforeUpdate such that zero values are only set when the
object does not have a zero value for the respective field.

@k8s-ci-robot k8s-ci-robot added size/L and removed lgtm size/M labels Aug 21, 2018

@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Aug 21, 2018

Contributor

/lgtm
/approve

Contributor

sttts commented Aug 21, 2018

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 21, 2018

@k8s-ci-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-ci-robot

k8s-ci-robot Aug 21, 2018

Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, nikhita, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Contributor

k8s-ci-robot commented Aug 21, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, nikhita, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 21, 2018

Member

/hold cancel
#67562 has merged

Member

nikhita commented Aug 21, 2018

/hold cancel
#67562 has merged

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Aug 26, 2018

Contributor

This PR is not for the master branch but does not have the cherrypick-approved label. Adding the do-not-merge/cherry-pick-not-approved label.

Contributor

k8s-merge-robot commented Aug 26, 2018

This PR is not for the master branch but does not have the cherrypick-approved label. Adding the do-not-merge/cherry-pick-not-approved label.

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 26, 2018

Member

The labels are messed up because of kubernetes/test-infra#9128. We need a prow bump to fix this.

Member

nikhita commented Aug 26, 2018

The labels are messed up because of kubernetes/test-infra#9128. We need a prow bump to fix this.

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 27, 2018

Member

Closing and opening the PR to trigger prow to remove the do-not-merge/cherry-pick-not-approved label.

Member

nikhita commented Aug 27, 2018

Closing and opening the PR to trigger prow to remove the do-not-merge/cherry-pick-not-approved label.

@nikhita nikhita closed this Aug 27, 2018

@nikhita nikhita reopened this Aug 27, 2018

@cjwagner

This comment has been minimized.

Show comment
Hide comment
@cjwagner

cjwagner Aug 27, 2018

Member

Take 2 now that the plugin is actually enabled.
/assign
/close

Member

cjwagner commented Aug 27, 2018

Take 2 now that the plugin is actually enabled.
/assign
/close

@cjwagner

This comment has been minimized.

Show comment
Hide comment
@cjwagner

cjwagner Aug 27, 2018

Member

/reopen

Member

cjwagner commented Aug 27, 2018

/reopen

@cjwagner

This comment has been minimized.

Show comment
Hide comment
@cjwagner

cjwagner Aug 27, 2018

Member

Woohoo!
/unassign

Member

cjwagner commented Aug 27, 2018

Woohoo!
/unassign

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Aug 28, 2018

Contributor

/test all [submit-queue is verifying that this PR is safe to merge]

Contributor

k8s-merge-robot commented Aug 28, 2018

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Aug 28, 2018

Contributor

Automatic merge from submit-queue.

Contributor

k8s-merge-robot commented Aug 28, 2018

Automatic merge from submit-queue.

@k8s-merge-robot k8s-merge-robot merged commit bd57e24 into kubernetes:release-1.11 Aug 28, 2018

17 of 18 checks passed

Submit Queue Required Github CI test is not green: pull-kubernetes-kubemark-e2e-gce-big
Details
cla/linuxfoundation nikhita authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Skipped
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-e2e-kubeadm-gce Skipped
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details

@nikhita nikhita deleted the nikhita:automated-cherry-pick-of-#67562-upstream-release-1.11 branch Aug 28, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment