pre-1.0 stability guarantees

[clux]

based on the client library support level doc from sig api-machinery we are currently at beta and have the current stable requirements:

• support level documented per-platform
• Library documentation - yes, via #[deny(missing_docs)]: https://github.com/clux/kube-rs/blob/38caa6beee3c7a12fc7bd46a5357e1258196cfca/kube/src/lib.rs#L74
• Deprecation policy (backwards compatibility guarantees documented)
  • How fast may the interface change?
• Release process well documented - yes, release.toml describes procedure, linked to in 38caa6b
• Versioning procedure well documented - one liner in about pre-1.0 semver strategy 38caa6b
• N documented users of the library - we could link to many: Stackable (all their operators) + Microsoft (krustlet, krator), logdna agent, rlink-rs, kubeapps proxy, perhaps also company concepts like h2oai + ephemeron, and perhaps cli tools like kdash - took a small subset in d8b42c7

Some key questions and observations:

how do we define how fast interfaces change?

obviously at post 1.0 we can deprecate fully, and parallel introduce features. but pre-1.0, things do change.
is there value in quantifying this? i know people will kind of look at these documents and use it as empty justifications to companies that's basically "hey, look, dd!". i joke, but like, how much value is there in saying "at most 2 breaking changes per release" other than it encouraging smaller releases (and we release pretty frequently in general). Maybe there are better ways to describe this?

deprecation policies

pre 1.0 i feel we can do a best effort to not break things, and where possible have deprecated functions available for at least 6 months. of course this won't always be possible yet; we don't have full feature parity with go, and the better design choices are not always clear. in those cases, we will denote changes as BREAKING in CHANGELOG.md and users can do changes at their own will through version pinning with cargo.

Per platform support

Well, we can document that we support Linux, and that everything else might require feedback for subtle issues. Windows / Mac are to some degree relying on strength-in-numbers atm because CI for these is in general not free.
What about Arm? We don't build for that here, but big community of tinkerers that try to build lightweight stuff on k3s.

definition of stability

an interesting consequence worth noting here:
because of the requirements in the linked document, we can have the kubernetes: stable badge even before our 1.0 release because it doesn't require us to never break things, it just requires us documenting our procedures/policies for breaking changes. Honestly, I think that's a pretty nice approach, even if it might look a bit strange to people seeing a stable badge on a pre-1.0 crate.

as a side-note following CloudNativeRustDay: people already consider us mature from many shoutouts and this on twitter: https://twitter.com/JamesLaverack/status/1389326493180903427

[clux]

also potentially worth mentioning is version compat with kubernetes, which to a large degree is solved by k8s-openapi version selection, but beyond that, we mostly operate on a "what's available at the 3 big cloud providers" basis.

client-go side-steps the "how fast" question, by introducing a compatibility matrix, we could go down that approach, but it feels a bit unnecessary because there's technically not a lot limiting us from being used on older kubernetes versions. it's only when you do api operations on older objects.

EDIT: Actually, we do make some decisions on versions. AdmissionResponse defaults to admission.k8s.io/v1beta1 when we fail to parse the review. If that becomes deprecated we do need to bump that.

[kazk]

Windows / Mac are to some degree relying on strength-in-numbers atm because CI for these is in general not free.

GitHub Actions supports them for free for open source projects. We can use strategy.matrix to run our tests in all supported OS in parallel. For example, I've been using it to publish binaries on release. https://github.com/qualified/lsp-ws-proxy/blob/b63f30f4200ab0e6bbd273d7599a64159a4a78fc/.github/workflows/upload-binaries.yml#L12

We should be able to use k3d or kind.

[clux]

A couple of proposals/discussion points for a classes of stability guarantees:

1. When we define traits, or re-export types that rely on specific kubernetes apis (e.g. CustomResourceExt via add a versioned CustomResourceExt trait - fixes #497 #545, admission module types), we keep the old versions available until they are dropped from major cloud providers. add a versioned CustomResourceExt trait - fixes #497 #545 provides one potential example of this type of versioned changes.

We can track disappearing apis through the deprecation guide and compare to min EKS versions + GKE stable channel, and maybe the AKS calendar.

We can introduce new versions as the main versions when the are available in major cloud providers. If we change what the main export is, then it will break compiles for people so it's a pretty clean way of having a main export that's the latest and GA version.

2. When we define generic implementations that rely on core concepts of the kubernetes apis (like the watch api (kube-runtime), or the meta::v1 (affecting Api and associated traits)), then we have to make a major version change to take advantage of it.

Not sure what would happen if kubernetes dares to break meta::v1, or the watch api, but if they are going to get away with it, it sure as hell needs to be worth it. So far they've only dared to add optional metadata types, and new enum types to watchevent (under feature flags and query params), so highly doubt they'll do something drastic at this point, but hey, they might want to clean up some of the messes they have.

[clux]

Another thing we could do is mark unstable apis like tokio does:

• https://docs.rs/tokio/1.12.0/tokio/#unstable-features
• https://github.com/tokio-rs/tokio/blob/f2a06bff1be147a72d40cb01d8bb621fbdc242fc/tokio/src/runtime/tests/mod.rs#L7-L18
• https://github.com/tokio-rs/tokio/blob/b5c1fb401249fac0d8c96dea1b52c096d26be8b9/tokio/Cargo.toml#L102-L105

[clux]

Closing this in favour of #726 which is a lot more elaborate and professional.