New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent resourceVersion updates for custom resources on no-op writes #67562

Merged
merged 1 commit into from Aug 21, 2018

Conversation

@nikhita
Member

nikhita commented Aug 18, 2018

Fixes partly #67541

For ObjectMeta pruning, we round trip by marshalling and unmarshalling. If the ObjectMeta contained any strings with "" (or other fields with empty values) and the respective fields are omitempty, those fields will be lost in the round trip process.

This makes ObjectMeta after the no-op write different from the one before the write.

Resource version is incremented every time data is written to etcd. Writes to etcd short-circuit if the bytes being written are identical to the bytes already present.

So this ends up incrementing the resourceVersion even on no-op writes. This PR updates the BeforeUpdate function such that omitempty fields have values set only if they are non-zero so that they produce an unstructured object that matches ObjectMeta omitempty semantics.

/sig api-machinery
/kind bug
/area custom-resources
/assign sttts liggitt

Release note:

Prevent `resourceVersion` updates for custom resources on no-op writes.
@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 18, 2018

Member

I'm wondering if this is fine to cherry-pick to 1.11 or not. If someone uses client-go to interact with unstructured objects, this could lead to breaking changes for them.

Member

nikhita commented Aug 18, 2018

I'm wondering if this is fine to cherry-pick to 1.11 or not. If someone uses client-go to interact with unstructured objects, this could lead to breaking changes for them.

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 18, 2018

Member

This additionally fixes two other issues as well: #48211 and #49075. 🎉

Member

nikhita commented Aug 18, 2018

This additionally fixes two other issues as well: #48211 and #49075. 🎉

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Aug 18, 2018

Member

I'm wondering if this is fine to cherry-pick to 1.11 or not. If someone uses client-go to interact with unstructured objects, this could lead to breaking changes for them.

It's important to resolve the incrementing resourceVersion issue for 1.11. Were the only fields force-set to an empty value in BeforeCreate/BeforeUpdate the clusterName and initializers fields? We may be able to make a more targeted fix for 1.11 to only force-set those fields to empty values if they contain non-empty values

Member

liggitt commented Aug 18, 2018

I'm wondering if this is fine to cherry-pick to 1.11 or not. If someone uses client-go to interact with unstructured objects, this could lead to breaking changes for them.

It's important to resolve the incrementing resourceVersion issue for 1.11. Were the only fields force-set to an empty value in BeforeCreate/BeforeUpdate the clusterName and initializers fields? We may be able to make a more targeted fix for 1.11 to only force-set those fields to empty values if they contain non-empty values

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 18, 2018

Member

We may be able to make a more targeted fix for 1.11 to only force-set those fields to empty values if they contain non-empty values

👍

Were the only fields force-set to an empty value in BeforeCreate the clusterName and initializers fields?

In BeforeCreate:

  1. Namespace
  2. DeletionTimestamp
  3. DeletionGracePeriodSeconds
  4. Initializers
  5. ClusterName

In BeforeUpdate:

  1. Namespace
  2. Initializers
  3. ClusterName

But I've noticed that we need to include selfLink and resourceVersion as well (they were creating problems in tests). I think this is because they are cleared prior to writing to etcd.

Member

nikhita commented Aug 18, 2018

We may be able to make a more targeted fix for 1.11 to only force-set those fields to empty values if they contain non-empty values

👍

Were the only fields force-set to an empty value in BeforeCreate the clusterName and initializers fields?

In BeforeCreate:

  1. Namespace
  2. DeletionTimestamp
  3. DeletionGracePeriodSeconds
  4. Initializers
  5. ClusterName

In BeforeUpdate:

  1. Namespace
  2. Initializers
  3. ClusterName

But I've noticed that we need to include selfLink and resourceVersion as well (they were creating problems in tests). I think this is because they are cleared prior to writing to etcd.

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 18, 2018

Member

I'll create a PR for 1.11 once this looks good to merge.

Member

nikhita commented Aug 18, 2018

I'll create a PR for 1.11 once this looks good to merge.

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

Updated. Ptal.

Member

nikhita commented Aug 20, 2018

Updated. Ptal.

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

/retest

Member

nikhita commented Aug 20, 2018

/retest

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

/lint

Member

nikhita commented Aug 20, 2018

/lint

@k8s-ci-robot

@nikhita: 2 warnings.

In response to this:

/lint

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Aug 20, 2018

Member

would it make sense to do the minimal fix with conditional zero-value sets in BeforeCreate/BeforeUpdate, along with the custom resource resourceVersion tests from this PR, so we can make sure that fixes the issue in a way we can safely pick to 1.11, then do these unstructured changes in a follow up? I'd like the 1.11 pick to be an actual pick of a master branch change if possible.

Member

liggitt commented Aug 20, 2018

would it make sense to do the minimal fix with conditional zero-value sets in BeforeCreate/BeforeUpdate, along with the custom resource resourceVersion tests from this PR, so we can make sure that fixes the issue in a way we can safely pick to 1.11, then do these unstructured changes in a follow up? I'd like the 1.11 pick to be an actual pick of a master branch change if possible.

@k8s-ci-robot k8s-ci-robot added size/M and removed size/L labels Aug 20, 2018

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

would it make sense to do the minimal fix with conditional zero-value sets in BeforeCreate/BeforeUpdate, along with the custom resource resourceVersion tests from this PR, so we can make sure that fixes the issue in a way we can safely pick to 1.11, then do these unstructured changes in a follow up? I'd like the 1.11 pick to be an actual pick of a master branch change if possible.

Done. Ptal.

Member

nikhita commented Aug 20, 2018

would it make sense to do the minimal fix with conditional zero-value sets in BeforeCreate/BeforeUpdate, along with the custom resource resourceVersion tests from this PR, so we can make sure that fixes the issue in a way we can safely pick to 1.11, then do these unstructured changes in a follow up? I'd like the 1.11 pick to be an actual pick of a master branch change if possible.

Done. Ptal.

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

/retest

Member

nikhita commented Aug 20, 2018

/retest

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

/retest

Member

nikhita commented Aug 20, 2018

/retest

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

/retest

Member

nikhita commented Aug 20, 2018

/retest

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

/retest

Member

nikhita commented Aug 20, 2018

/retest

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Aug 20, 2018

Member

/lgtm

Member

liggitt commented Aug 20, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 20, 2018

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Aug 20, 2018

Member

thanks, can you pick this to 1.11 once it merges?

Member

liggitt commented Aug 20, 2018

thanks, can you pick this to 1.11 once it merges?

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Aug 20, 2018

Member

/assign @sttts

Member

liggitt commented Aug 20, 2018

/assign @sttts

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 20, 2018

Member

thanks, can you pick this to 1.11 once it merges?

will do. also, I'll create a PR against master with the remaining changes.

Member

nikhita commented Aug 20, 2018

thanks, can you pick this to 1.11 once it merges?

will do. also, I'll create a PR against master with the remaining changes.

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Aug 20, 2018

Member

/hold

actually, I think the same change needs to be made in BeforeCreate(), otherwise a create immediately followed by a no-op update would still modify what is stored in etcd

Member

liggitt commented Aug 20, 2018

/hold

actually, I think the same change needs to be made in BeforeCreate(), otherwise a create immediately followed by a no-op update would still modify what is stored in etcd

@k8s-ci-robot k8s-ci-robot added do-not-merge/hold and removed lgtm labels Aug 20, 2018

@lavalamp

This comment has been minimized.

Show comment
Hide comment
@lavalamp
Member

lavalamp commented Aug 20, 2018

/assign @roycaihw @jpbetz

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Aug 20, 2018

Member

/hold cancel
/lgtm

Member

liggitt commented Aug 20, 2018

/hold cancel
/lgtm

@k8s-ci-robot k8s-ci-robot added lgtm and removed do-not-merge/hold labels Aug 20, 2018

@roycaihw

This comment has been minimized.

Show comment
Hide comment
@roycaihw

roycaihw Aug 20, 2018

Member

/lgtm

Member

roycaihw commented Aug 20, 2018

/lgtm

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 21, 2018

Member

/retest

Member

nikhita commented Aug 21, 2018

/retest

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 21, 2018

Member

For the record:

  • #67616: cherry-pick of this PR to 1.11
  • #67635: follow-up PR against master containing fixes for unstructured accessors
Member

nikhita commented Aug 21, 2018

For the record:

  • #67616: cherry-pick of this PR to 1.11
  • #67635: follow-up PR against master containing fixes for unstructured accessors
Prevent resource version to custom resource on no-op writes
For ObjectMeta pruning, we round trip through marshalling and
unmarshalling. If the ObjectMeta contained any strings with "" (or other
fields with empty values) _and_ the respective fields are omitempty,
those fields will be lost in the round trip process.

This makes ObjectMeta after the no-op write different from the one
before the write.

Resource version is incremented every time data is written to etcd.
Writes to etcd short-circuit if the bytes being written are identical
to the bytes already present. So this ends up incrementing the
resourceVersion even on no-op writes.

The zero values are set in BeforeCreate and BeforeUpdate. This commit
updates BeforeUpdate such that zero values are only set when the
object does not have a zero value for the respective field.
@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 21, 2018

Member

Picked the test changes from @sttts's PR: #67637 (as discussed with @sttts on slack on the #sig-api-machinery channel).

Member

nikhita commented Aug 21, 2018

Picked the test changes from @sttts's PR: #67637 (as discussed with @sttts on slack on the #sig-api-machinery channel).

@sttts

This comment has been minimized.

Show comment
Hide comment
@sttts

sttts Aug 21, 2018

Contributor

/lgtm
/approve

Contributor

sttts commented Aug 21, 2018

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 21, 2018

@k8s-ci-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-ci-robot

k8s-ci-robot Aug 21, 2018

Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, nikhita, roycaihw, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Contributor

k8s-ci-robot commented Aug 21, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, nikhita, roycaihw, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nikhita

This comment has been minimized.

Show comment
Hide comment
@nikhita

nikhita Aug 21, 2018

Member

/retest

Member

nikhita commented Aug 21, 2018

/retest

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Aug 21, 2018

Contributor

/test all [submit-queue is verifying that this PR is safe to merge]

Contributor

k8s-merge-robot commented Aug 21, 2018

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Aug 21, 2018

Contributor

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

Contributor

k8s-merge-robot commented Aug 21, 2018

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-merge-robot k8s-merge-robot merged commit 816f2a4 into kubernetes:master Aug 21, 2018

17 of 18 checks passed

Submit Queue Required Github CI test is not green: pull-kubernetes-kubemark-e2e-gce-big
Details
cla/linuxfoundation nikhita authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-e2e-kubeadm-gce Skipped
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details

k8s-merge-robot added a commit that referenced this pull request Aug 22, 2018

Merge pull request #67635 from nikhita/customresource-subresource-pat…
…ch-04

Automatic merge from submit-queue (batch tested with PRs 67298, 67518, 67635, 67673). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix unstructured metadata accessors to respect omitempty semantics

Fixes #67541
Fixes #48211 
Fixes #49075
Follow up of #67562

`ObjectMeta` has fields with `omitempty` json tags. This means that when the fields have zero values, they should not be persisted in the object.

Before this PR, some of the metadata accessors for unstructured objects did not respect these semantics i.e they would persist a field even if it had a zero value.

This PR updates the accessors so that the field is removed from the unstructured object map if it contains a zero value.

/sig api-machinery
/kind bug
/area custom-resources
/cc sttts liggitt yue9944882 roycaihw 
/assign sttts liggitt 

**Release note**:

```release-note
NONE
```

@nikhita nikhita deleted the nikhita:customresource-subresource-patch branch Aug 22, 2018

k8s-merge-robot added a commit that referenced this pull request Aug 28, 2018

Merge pull request #67616 from nikhita/automated-cherry-pick-of-#67562-…
…upstream-release-1.11

Automatic merge from submit-queue.

Automated cherry pick of #67562: Prevent resourceVersion updates for custom resources on no-op writes

Cherry pick of #67562 on release-1.11.

#67562: Prevent resourceVersion updates for custom resources on no-op writes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment