-
Notifications
You must be signed in to change notification settings - Fork 39k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The metadata.generation of a Custom Resource is always incremented #69059
The metadata.generation of a Custom Resource is always incremented #69059
Conversation
/kind bug |
@@ -87,7 +87,7 @@ func (a customResourceStrategy) PrepareForCreate(ctx context.Context, obj runtim | |||
|
|||
// PrepareForUpdate clears fields that are not allowed to be set by end users on update. | |||
func (a customResourceStrategy) PrepareForUpdate(ctx context.Context, obj, old runtime.Object) { | |||
if !utilfeature.DefaultFeatureGate.Enabled(apiextensionsfeatures.CustomResourceSubresources) || a.status == nil { | |||
if !utilfeature.DefaultFeatureGate.Enabled(apiextensionsfeatures.CustomResourceSubresources) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A CR can have the .status field, while opted-out of the /status endpoint. User would expect the metadata.generation to auto-increment.
My understanding is that .metadata.generation
should be incremented only when the user specifically says that they want to follow spec/status conventions - which means they need to explicitly set the .subresources.status
field in the CRD and opt-in to /status
. Enabling the feature gate does not guarantee a status subresource because it could also mean that they only want to enable /scale
.
Related: #58778 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I almost forgot I had this early involvement in the 58778 :)
I think our disagreement is what signals the CR is following the spec/status convention. I think a CR that has a top level spec
and status
field is ready to follow the spec/status convention. But the current code assumes that's not enough. The current code assumes the signal is the enablement of the /status
subresource.
Taking one step back, what's wrong if the generation is always auto-incremented, even if the CR doesn't follow the spec/status convention? The field isn't writable by users anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@caesarxuchao Do you have a use case in mind for wanting generation
to increment even if the CRD author did not explicitly opt into the full spec/status convention (by enabling /status subresource)?
We intentionally chose not to infer that the CRD author wants to use the spec/status convention just because they have fields named spec
and/or status
. That would make it impossible to use those field names without participating in the convention, which would be a backwards-incompatible change.
Or are you just saying it's a confusing UX that you have to turn on a subresource to opt into the spec/status convention? If that's the case, perhaps there could be a separate field to enable generation
for spec
without enabling the /status
subresource, with validation that you can't have the subresource without having generation
too. I'm not aware of a strong use case for wanting those things separately, though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe users just want to see how many times the spec
has been changed.
The explanation in the second paragraph makes sense.
It's a confusing UX that I have to turn on a subresource to make apiserver manages my generation
. Adding a specific field to turn on generation management is better than the status quo. But why is it bad to always let the apiserver manage the generation field as long as the CR has the spec
field?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I almost forgot I had this early involvement in the 58778 :)
:)
If that's the case, perhaps there could be a separate field to enable generation for spec without enabling the /status subresource, with validation that you can't have the subresource without having generation too.
IMHO adding a new field is too much complexity for this.
But why is it bad to always let the apiserver manage the generation field as long as the CR has the spec field?
Thinking about this more, this makes sense but I still have one concern.
The current behaviour is to increment the generation if the user opts in to /status
i.e. they opt-in to spec and status conventions. We did this because the general expectation is that metadata.generation
will be useful in a controller with a corresponding status.observedGeneration
(which implies that the user should also opt-in to status). This is also what #58778 (comment) said.
But metadata.generation
is technically a "per object sequence number" (taking the terminology from #7328). The sequence number of an object should not depend on the presence of status.observedGeneration
. It should increment with every spec change, irrespective of whether it will prove useful.
This means that we just require the user to opt-in to spec conventions. Opting in to status conventions should not be a requirement. .spec
holds special meaning for both /status
and /scale
. So, if one of these is enabled, I think we should increment generation.
My only concern is that just enabling the feature without opting in to /status
or /scale
doesn't guarantee that I want to follow spec conventions. Right now, enabling the feature gate does not hold much meaning unless the user specifically opts-in to either one of the subresources.
That would make it impossible to use those field names without participating in the convention, which would be a backwards-incompatible change.
With this change, we redefine the feature gate to also mean that .spec
will participate in the spec conventions. I'm concerned that it will be a backwards-incompatible change.
(But this doesn't really help your use case :/)
/area custom-resources |
06923f0
to
2a20fa7
Compare
I agree that the RE. backwards compatibility, I think unconditional increment is ok, because I can't find any reason why a user would depend on |
/retest |
I could buy that it makes sense to always increment The API reference defines
If a resource does not participate in the spec/status convention, which for backwards compatibility we have to assume if they do not explicitly opt in, then the default meaning of "desired state" is the entire object. The only reason we treat |
@enisoc, I agree with your reasoning in #69059 (comment). However, I'm concerned with UX. The effect of |
Even if you didn't need it, or if a hypothetical user doesn't know they need it, we wanted to force you to have a /status subresource before taking any action that would imply you have spec/status support in your custom resource. One of our design goals was to treat "spec/status convention" as a holistic set of behaviors, which includes a /status subresource and special treatment of metadata.generation (modifying the default assumption that the whole object is part of "desired state"). We wanted to discourage people from opting in to only bits and pieces of the overall convention. In retrospect, maybe we should have made it controlled by a top-level field like
You're much more familiar with the inner workings of core controllers, though. If I try to imagine I'm a typical user, and I see this is how
I would argue that typical user would be surprised if certain fields were treated specially by default. There's no mention in that definition of |
We don’t require that all objects have spec and status, nor do we require that status be paired with spec, but we very strongly suggest both. However, we do not bump generation when metadata is changed, so neither labels and annotations can be input to generation, even though in practice many controllers may rely on that action. I would probably be inclined to have generation be bumped on CRDs be bumped when any non metadata field is updated, and if status is defined, any non-metadata, non-status field is updated. |
2a20fa7
to
ed87cf3
Compare
@enisoc PTAL. The new behavior is close to what you and Clayton described. Thanks. |
ed87cf3
to
ac7056a
Compare
/retest |
1 similar comment
/retest |
The failed integration test has nothing to do with this PR:
I'll retry later. |
@enisoc I updated the code to what you suggested, PTAL. Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
incrementGeneration = true | ||
} | ||
|
||
if incrementGeneration { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: The bool var doesn't seem necessary anymore. Can't you just put this block under if !DeepEqual
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Mind applying the lgtm again? Thanks.
2341966
to
88bb53f
Compare
/lgtm |
/retest |
if (ok1 && ok2 && !apiequality.Semantic.DeepEqual(oldSpec, newSpec)) || (ok1 && !ok2) || (!ok1 && ok2) { | ||
// except for the changes to `metadata`, any other changes | ||
// cause the generation to increment. | ||
newObjCopy := newCustomResourceObject.DeepCopy() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no need for a deepcopy here. Do a shallow-copy of the top-level map instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" | ||
) | ||
|
||
func generation1() map[string]interface{} { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wow, two 5 line helpers for expressions that are one-liners: map[string]interface{}{"generation": int64(1)}
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would trade 15 long one-liners with 2 five-line helpers :)
Looks good overall. Some nits. |
If the custom resource participates the spec/status convention, the metadata.generation of the CR increments whenever there is any change, except for the changes to the metadata or the changes to the status. If the CR does not participate the spec/status convention, the metadata.generation of the CR increments whenever there is any change, except for the changes to the metadata. A CR is considered to participate the spec/status convention if and only if the "CustomResourceSubresources" feature gate is turned on and when the CRD has `.spec.subresources.status={}`.
88bb53f
to
417db5f
Compare
Lgtm from my side. Can't tag because GH is down. |
/lgtm |
@sttts can you approve? |
@sttts can you approve? Thanks. |
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: caesarxuchao, sttts The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
The old behavior is that the
metadata.generation
never changes if the user doesn't participate the spec/status convention. See the release note for the new behavior./assign @nikhita @mbohlool
/sig api-machinery