Adding GEP 922: Gateway API Versioning Plan

[robscott]

What type of PR is this?
/kind gep

What this PR does / why we need it:
This proposes a set of versioning guidelines for future Gateway API releases. This is a follow up to the related doc that has been discussed in gateway-api and sig-apimachinery meetings.

Which issue(s) this PR fixes:
Related to #922, will not be fixed until this GEP moves to "Implemented"

Does this PR introduce a user-facing change?:

NONE

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: robscott

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [robscott]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[youngnick]

This is looking pretty good overall to me, but there's a couple of things that could be clearer.

[hbagdi]

So, at the end of the day, a cluster can either have a stable or an experimental CRD. It is exclusive because you can only have one CRD at a time for a GK.

Why have the distinction in that case?
What if we just ship the experimental fields as part of the stable channel?

[youngnick]

If we do that, then the only way for YAML consumers (and let's be real, most people will use YAML, not read the full spec) to know what fields are experimental is for us to prefix them somehow - that is, we'd have to call all the experimental fields experimentalFieldName or something.

Other things I can think of that only supplying on CRD spec would imply:

• Implementations would need a switch of some sort to turn on experimental field processing - effectively, we're moving the feature gate to the implementing controller, and trusting them to figure it out.
• We'd also need to supply a method for implementations to determine what the experimental fields are, so that they can tell people they're using an experimental field but haven't enabled it.

Having two separate CRDs solves all of those problems - the implementation can switch experimental field processing on or off based on the annotation on the CRD object itself (this also means you can't enable experimental field processing without having modify CRDs access, which I think is fine).

[robscott]

Agree with @youngnick's points above. I think it's important to have users explicitly opt in to experimental functionality, and I think separate CRDs are likely the best mechanism we have for this. Although I'd really like something better like kubectl support/gating for alpha fields, I don't think that's going to be practical anytime soon.

[hbagdi]

I totally see the need for doing this within the same API version but I also think it is our responsibility to help out implementers here. Is there any way a controller can check the version bundle version in the cluster?
Like an annotation on the CRD (I'm not sure if those annotations are readable programmatically from a controller or not).

Since Gateways are very sensitive to security, most operators will prefer a hard failure compared to an implementation having buggy behavior (or a vulnerable behavior). We don't need to instruct what controller authors should do when the bungle version is higher than the controller is programmed against.

Can we make this part of this GEP?

[robscott]

Good point - we do have annotations proposed further down that would enable this. Annotations should be able to be read programmatically, as long as the controller requests read access on CRDs which seems reasonable and likely necessary.

[hbagdi]

What if an implementation doesn't want to work in a cluster unless the installed version matches a support matrix that the it is programmed against? Is that invalid behavior?

[robscott]

I think that's valid, but likely unnecessarily constrained. If all the guidelines here are followed, newer additions to the API would simply not be supported by an older implementation of the API. A warning or set of warnings is likely preferable to stopping altogether.

[hbagdi]

Okay. Can we explicitly mention that controllers can rely on the annotation to decide their behavior?

[youngnick]

This lgtm with a few non-blocking nits.

[youngnick]

Should we mention other validation code here?

[robscott]

I think we should definitely add that to our bundle versions if/when we have it. I have a note in #926 to come back to this KEP and update it if/when we add that logic.

[youngnick]

Suggested change

	* MUST provide conversion between stable API versions, starting with v1alpha2.
	* MUST provide conversion between API versions, starting with v1alpha2.

Feels weird to call an alpha API "stable".

[robscott]

Yeah, the terminology here is confusing. I've updated this to "excluding experimental fields", which I think is a clearer representation of what I was aiming for here.

[youngnick]

This wording is fine for now, but if we do implement the validation module, feels like it should be a MUST to me.

[robscott]

Good catch, I added a note to #926 so we remember to come back to this GEP and update it.

[youngnick]

/lgtm
/hold for multiple lgtm though.

[hbagdi]

Here is to hoping we have thought this through.
/lgtm

[robscott]

I think there's broad enough consensus on this to merge it in, we can revisit through smaller tweaks if needed.

/hold cancel