Validation Tightening is not Generally Possible

[lavalamp]

I have made this comment on a few PRs so maybe it's time to document and/or build a mechanism.

You cannot generally make validation tighter, because any existing objects that violate the new rule will not be able to be updated, and in many cases, deleted (because deletions involving finalizers require a sequence of updates).

This is because, to validate updates, we validate BOTH the transition, AND the final state of the object. It's this last part that is problematic.

It should be possible to address this by adding a new class of validation function that ONLY gets called on new objects, but then we have these problems:

• When is it safe to promote the logic into the main validation function? There's no way to know.
• This solves the problem for apiserver, but not for any clients that were sending invalid objects. Especially automated clients will suddenly and without recourse be quite broken.

Even at version boundaries it's hard to tighten validation, since objects have to be round-trippable and there are still old objects in the system.

So, new API authors: start out with restrictive validation; it's easier to relax than to tighten.

Existing API authors: prepare to write defensive clients.

/kind bug
/sig api-machinery

[liggitt]

xref #59984 #60635 #57615 #52936 #59593 #60934 #61623 #61984 #62987 #63426 #65470 for wanting to tighten validation

wanting to loosen validation:

• [Need-Feedback] Support Volume Expansion Through StatefulSets #71384 (comment)
• Added ':' in environment variable name valid characters list #69415 (comment)
• Add StatefulSet Volume Expansion Kep enhancements#660 (comment)

[liggitt]

a methodical way to do ratcheting validation would be ideal:

• apply tightened validation on create
• apply tightened validation on update where the old object passed this validation (don't let a valid object become invalid)
• skip tightened validation on update where the old object fails this validation

[lavalamp]

Yes, that's the mechanism I had in mind when I wrote the two bullet points in the original post.

(also thanks for the extensive list of cross references)

[jennybuckley]

/cc @mbohlool

[liggitt]

/assign

[liggitt]

WIP in #64907

[tallclair]

xref #64532

[bgrant0607]

Sorry for not noticing this earlier.

Tightening validation can break clients. I don't see much of a way around that. Special-casing updates protects the apiserver but not clients and users.

It seems like the only recourse would be to downgrade to the prior release until the clients can be fixed. And downgrade is known to not work in the general case.

Maybe we need an advisory mode that also applies to create, so that users can debug problems before they irreversibly hose their clusters?

[bgrant0607]

https://github.com/kubernetes/community/blob/master/contributors/devel/api_changes.md#backward-compatibility-gotchas

Changing any validation rules always has the potential of breaking some client, since it changes the assumptions about part of the API, similar to adding new enum values. Validation rules on spec fields can neither be relaxed nor strengthened. Strengthening cannot be permitted because any requests that previously worked must continue to work. Weakening validation has the potential to break other consumers and generators of the API resource. Status fields whose writers are under our control (e.g., written by non-pluggable controllers), may potentially tighten validation, since that would cause a subset of previously valid values to be observable by clients.

[bgrant0607]

The end of that should be: "to no longer be observable by clients"