[RFD] API Versioning

[jsollom-hpe]

API Versioning

What needs to change:

In the past, APIs have been versioned as a whole where the version encompasses all of the endpoints. This leads to a monolithic view of the API preventing it from being more composable. Changes in one resource require bumping the global API version. There is potential for more frequent version churn. We should move away from versioning the entire API.

What do you propose?

Versioning will be done on a resource basis. Each resource-collection will be versioned as opposed to the entire API being versioned. This composability allows changes to individual resources without affecting the remainder of the API.

The version will precede the resource-collection. The URI syntax is as follows:

/api-group/version/resource-collection/

Examples:
GET /storage/v1/storage-systems

GET /storage/v2/storage-systems

The Google AIP-185 API Versioning puts the major version number in the URI. The use of the term "major version number" is taken from semantic versioning at semver.org. However, unlike in traditional semantic versioning, Google APIs must not expose minor or patch version numbers. For example, Google APIs use v1, not v1.0, v1.1, or v1.4.2. From a user's perspective, major versions are updated in place, and users receive new functionality without migration. For this reason, this RFD suggests only using the major version number in the URL.

When incrementing the version on a resource-collection, the sunset HTTP header field could be used to indicate when a resource-collection is being deprecated.
See this standard for IETF RFC 8594 The Sunset HTTP Header Field.

What alternatives exist?

The version could be specified in the request HEADER.
Example: GET /users with header Accept: application/vnd.myapi.v2+json

Pros:
• Clean URLs.
• Can support multiple representations or minor variations.
• Encouraged in REST-style APIs.

Cons:
• Harder for humans to discover.
• Not cache-friendly.
• Poor support in some tools or proxies.

Reference: IETF RFC 7231 - Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content, specifically the Accept header

This Github API Versions document describes an alternate method for supplying the version in the header.

Related RFDs

• Establishment of an API contract specification for OpenCHAMI
  • This RFD should be part of the API contract specification.

[alexlovelltroy]

I'm in favor of the idea of reducing the scope of a breaking change to a subset of a microservice, but I'm a little confused about how that would work in practice. Can you take something concrete and propose how you would see it evolving? Perhaps a subset of the SMD API for example? Some OpenAPI docs and/or pseudocode would be helpful to explore the edge cases that a standard like this would introduce.

[jsollom-hpe]

Here are examples of endpoints versioned by their resource-collection.

Versioning Format:

• /State//Components
• /Defaults//NodeMaps

They could both start at version 1, v1.

• /State/v1/Components
• /Defaults/v1/NodeMaps

Later, they can be independently incremented without affecting the entire API.

Later version example:

• /State/v5/Components
• /Defaults/v2/NodeMaps

Such a change to SMD would be a breaking change as these endpoints had not been previously versioned in this fashion. I am doing this for exemplary purposes, only.

[davidallendj]

Later version example:

* /State/v5/Components
* /Defaults/v2/NodeMaps

Does this mean I would have 5 implementations of Components and 2 implementations of NodeMaps in SMD's internal API? I can imagine our code bases becoming bloated, confusing, and much harder to manage with this approach. I'm going to assume part of the rationale here for the REST API versioning is to have correspondence with changes done to internal interfaces.

For example, if I have a function like doSomethingGet that uses Component and later create a ComponentV2, then I would also need to create a doSomethingGetV2 which may map to the REST API as /State/v1/Components and /State/v2/Components. If that's the case, when exactly would I use /State/v1/Components opposed to /State/v2/Components if we're just updating all of our microservices to work with the latter? If I hypothetically have 5 implementations, why would I want to manage converting between the implementations, if instead, I can just have one that's defined somewhere that all of my microservices already know about?

Ultimately, I'm wondering why we would want to go about handling versioning this way and/or if this would just balloon the amount of technical debt we'd have managing our APIs. I recall seeing some form of versioning within SMD's internal API itself (not necessarily the API), but I think we ended up only using one version anyways while having to make a couple of conversions here and there.

[jsollom-hpe]

I don't see this as being any worse than revving the API version 7 times to support 5 component changes and 2 NodeMap changes.

We could decide to only support so many revisions back of a given resource in a similar fashion to only supporting the previous release. I'm not sure I endorse that option because it would make older clients inoperable with newer servers.

The golden-handcuffs of backwards compatibility are a pain no matter how you version.

[rainest]

Minor bit: the services not mounting their API at / is kinda annoying, and different bits don't actually agree on the mounts or who's responsible for what.

I keep forgetting what the individual service current versions and mount points are (though this is maybe more indicative of me having to jump between too many different projects than it is a design flaw), and the swagger.yamls aren't necessarily correct.

BSS, for example, lists that it's version 1 with a root at /apis/bss, and SMD lists its v2 with a root at /apis/smd/hsm/v2. The actual roots in the code are instead /boot/v1/ and /hsm/v2/.

IDK if there's any service that actually exposes more than one API group+version, i.e. I don't think smd also supports a /hsm/v1/ or /othersmdapi/v2/; they only have the one root.

The ochami CLI could potentially support multiple versions, but at present it has no version knowledge AFAIK: it's compatible with only one version of a given API per release, and it does not insert the version path segments for you. You don't set bss.uri: http://example.com/ or bss.uri: http://example.com/boot/, you have to set it to bss.uri: http://example.com/boot/v1; it will not insert the relevant API or version path prefixes for you.

A given service release mounting its one and only API at its one and only version at / would be simplest, but I expect we will want releases that support multiple for smoother transitions. IDK if we can avoid services that need to support multiple APIs, but I think OpenCHAMI/cloud-init#71 informally moved towards separate API -> separate service, bring your own reverse proxy if you want them sharing a hostname.

For versions, I think it'd make sense for clients to inject the path segment (maybe with an override absolute URL that omits it), since they should have a better idea of which versions they support than their users.

[alexlovelltroy]

The more I think about this, the more I think it approaches the convention (standard?) for CRDs in kubernetes. We can draw inspiration from their experience with references below. While our architectures differ, Kubernetes’ experience scaling from a handful of built-in types to an open ecosystem of arbitrary resources is relevant. Their model supports resource evolution while maintaining backward compatibility, which is exactly the tension we face in OpenCHAMI.

Things to learn from:

1. API Groups
   In Kubernetes, API groups extend the API at path levels:

/apis/<GROUP_NAME>/<VERSION>

and are referenced internally via <GROUP_NAME>/<VERSION>.

In OpenCHAMI, HAProxy already plays the role of grouping, making multiple microservices appear to be one API.
If we treat smd as a single API group, the external structure could look like:

https://openchami.system.site/apis/smd/v2/

This would allow us to reference smd/v2 as a unit in documentation and releases, and to advertise support for specific group versions per deployment. Internally, smd itself would still route requests via /v2 in its own API router.

2. Integers vs alphanumeric for versions

Kubernetes distinguishes stable vs. unstable APIs by version naming:

• v1, v2 → stable
• v2beta3 → beta (pre-release for v2)
• v3alpha1 → alpha (experimental)

We currently only use whole integers. Borrowing Kubernetes’ pattern would let us signal stability to clients, e.g.:

• smd/v3beta1 → try it now, but expect breaking changes before v3
• smd/v3alpha1 → highly experimental

3. Resource-Level Versioning

Kubernetes versioning happens at the resource schema level as well as the API group version.

For example:

• The API group might be /apis/apps/v1
• Within it, a Deployment resource might have multiple served versions (v1, v1beta1) with one storage version.

Right now, a change to the Component resource in smd replacing id (currently an xname) with an opaque identifier and moving xname into location would force us to think in terms of a new API URI version.

Instead, we can version resource schemas within a group api version.

• Keep the group at smd/v2
• Internally serve Component in both v2 (old schema) and v3beta1 (new schema) for a period.
• Persist everything in the new schema, converting for old clients as Kubernetes does for CRDs.

This separation of API group version (what’s in the URI) from resource schema version (what’s in the payload) would give us more flexibility to evolve individual resources without churning the entire group version.

Code Example

package main

import (
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"strings"
	"sync"
)

// ---------- Resource versions ----------

// Legacy schema (clients still use this)
// apiVersion: smd.openchami.io/v2
type ComponentV2 struct {
	APIVersion string `json:"apiVersion"` // "smd.openchami.io/v2"
	Kind       string `json:"kind"`       // "Component"
	ID         string `json:"id"`         // xname-as-id (legacy)
	// ... other legacy fields ...
}

// New/storage schema (what we persist)
// apiVersion: smd.openchami.io/v3beta1
type ComponentV3 struct {
	APIVersion string          `json:"apiVersion"` // "smd.openchami.io/v3beta1"
	Kind       string          `json:"kind"`       // "Component"
	ID         string          `json:"id"`         // opaque id
	Location   ComponentLoc    `json:"location"`   // holds xname
	// ... evolved fields ...
}
type ComponentLoc struct {
	XName string `json:"xname"`
}

// ---------- Converters (hub-and-spoke style) ----------

func (v2 *ComponentV2) ToV3(storageID string) ComponentV3 {
	// storageID is the opaque id we mint/lookup for this xname
	return ComponentV3{
		APIVersion: "smd.openchami.io/v3beta1",
		Kind:       "Component",
		ID:         storageID,
		Location:   ComponentLoc{XName: v2.ID},
	}
}

func (v3 *ComponentV3) ToV2() ComponentV2 {
	return ComponentV2{
		APIVersion: "smd.openchami.io/v2",
		Kind:       "Component",
		ID:         v3.Location.XName, // surface xname as "id" for legacy clients
	}
}

References:

• https://github.com/kubernetes/sig-release/blob/master/release-engineering/versioning.md
• https://kubernetes.io/docs/reference/using-api/?utm_source=chatgpt.com#api-versioning

[njones-lanl]

Doing a Kube style API sounds reasonable to me given the problem constraint of having drifting inputs. Would we make a presumption if no API is specified, to use the API that exists currently, to not break version compatibility?

[haroldlongley]

These comments about Kubernetes APIs (previous comment) also pertain to the Google APIs referenced in this RFD description. Google AIP-185 API Versioning.

Kubernetes distinguishes stable vs. unstable APIs by version naming:

v1, v2 → stable
v2beta3 → beta (pre-release for v2)
v3alpha1 → alpha (experimental)

The description in GOOGLE AIP-185 continues past the use of the major version number from semver.org being used (v1 or v2) and not the minor (v2.2) or patch release (v1.4.2) numbers to discuss Channel-based versioning versus Release-based versioning.

Channel-based versioning:

A stability channel is a long-lived release at a given stability level that receives in-place updates. There is no more than one channel per stability level for a major version. Under this strategy, there are up to three channels available: alpha, beta, and stable.

The alpha and beta channel must have their stability level appended to the version, but the stable channel must not have the stability level appended. For example, v1 is an acceptable version for the stable channel, but v1beta or v1alpha are not. Similarly, v1beta or v1alpha are acceptable versions for the respective beta and alpha channel, but v1 is not acceptable for either. Each of these channels receives new features and updates "in-place".

The beta channel's functionality must be a superset of the stable channel's functionality, and the alpha channel's functionality must be a superset of the beta channel's functionality.

And

Release-based versioning:

An individual release is an alpha or beta release that is expected to be available for a limited time period before its functionality is incorporated into the stable channel, after which the individual release will be shut down. When using release-based versioning strategy, an API may have any number of individual releases at each stability level.

Note: Both the channel-based and release-based strategies update the stable version in-place. There is a single stable channel, rather than individual stable releases, even when using the release-based strategy.

Alpha and beta releases must have their stability level appended to the version, followed by an incrementing release number. For example, v1beta1 or v1alpha5. APIs should document the chronological order of these versions in their documentation (such as comments).

Each alpha or beta release may be updated in place with backwards-compatible changes. For beta releases, backwards-incompatible updates should be made by incrementing the release number and publishing a new release with the change. For example, if the current version is v1beta1, then v1beta2 is released next.

Alpha and beta releases should be shut down after their functionality reaches the stable channel. An alpha release may be shut down at any time, while a beta release should allow users a reasonable transition period; we recommend 180 days.

[larry-kaplan]

From the 2025 Summit discussion:
Need an ADR that recommends the Kubernetes versioning scheme with release based versioning.