Proposal: Versioning of provider artifacts and provider resources

[asilverman]

TL;DR

• We propose to relax the constraint on provider import declaration so that they are not required to conform to the format semantic versioning.

• We propose that each provider should define their version scheme and allow any valid OCI tag descriptor as a version value.

• We propose that provider author shall determine how the resource types are version descriptors are resolved.

• We believe that doing the above will improve the user experience by making it self-evident for the user what types are loaded from a provider declaration syntax.

Overview

The current implementation of Bicep describes the ability to import type namespaces and Bicep extensibility providers using the following syntax per the Bicep documentation: Import Bicep namespaces

import 'az@1.0.0-alpha.1'
import 'sys@1.0.0'
import 'kubernetes@1.0.0-alpha.1+12345458663-22314654asasa'

The syntax object used to parse the statements above constrains the version to the right of the '@' character to be a semantic version string format as enforced by the code below.

bicep/src/Bicep.Core/Syntax/ImportSpecification.cs

Lines 23 to 30 in ac299a3

	// Regex copied from https://semver.org/.
	// language=regex
	private const string SemanticVersionPattern = @"(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?";
	private static readonly Regex SpecificationPattern = new(
	@$"^(?<name>{NamePattern})@(?<version>{SemanticVersionPattern})$",
	RegexOptions.ECMAScript | RegexOptions.Compiled);

bicep/src/Bicep.Core/Syntax/ImportSpecification.cs

Lines 65 to 78 in ac299a3

	private static (string Name, string Version, bool IsValid) Parse(string value)
	{
	var match = SpecificationPattern.Match(value);
	if (!match.Success)
	{
	return (LanguageConstants.ErrorName, LanguageConstants.ErrorName, false);
	}
	var name = match.Groups["name"].Value;
	var version = match.Groups["version"].Value;
	return (name, version, true);
	}

We propose to relax the enforcement of the import version and delegate the concern of versioning to provider developers, in the sections below we will provide the motivation for doing this and how this change results in a better overall user experience for resource type provider authors and Bicep file creators alike.

We propose the language enforcement of the ImportSpecification string be relaxed to match a valid OCI image tag so that authors have more flexibility in delivering their package contents to users.

Motivation

In the following sections we will describe the limitations with the current design using examples.

Example 1: The Azure provider

In the import statement currently used to load the azure types provider (import 'az@1.0.0') the version ('1.0.0') is a placeholder. The versioning of the types is reflected in the NuGet package reference version (from Bicep.Core.csproj), as shown below:

bicep/src/Bicep.Core/Bicep.Core.csproj

Line 43 in ac299a3

<PackageReference Include="Azure.Bicep.Types.Az" Version="0.2.552" />

The definitions are generated via a console application (azure/bicep-types-az ) that is triggered manually for each Bicep release. The NuGet package version is then updated to reflect that the new types are a result of that generation event.

Inspection of the underlying types reveals that they follow the Azure REST API versioning specification, this specification defines a version as a date (e.g. 2023-11-01-preview, 2023-01-10).

bicep/src/Bicep.Core.Samples/Files/baselines/AKS_LF/main.bicep

Lines 27 to 56 in 7524201

	resource aks 'Microsoft.ContainerService/managedClusters@2020-03-01' = {
	name: clusterName
	location: location
	properties: {
	dnsPrefix: dnsPrefix
	agentPoolProfiles: [
	{
	name: 'agentpool'
	osDiskSizeGB: osDiskSizeGB
	vmSize: agentVMSize
	osType: 'Linux'
	storageProfile: 'ManagedDisks'
	}
	]
	linuxProfile: {
	adminUsername: linuxAdminUsername
	ssh: {
	publicKeys: [
	{
	keyData: sshRSAPublicKey
	}
	]
	}
	}
	servicePrincipalProfile: {
	clientId: servcePrincipalClientId
	secret: servicePrincipalClientSecret
	}
	}
	}

This creates a challenge for our customers since any semantic version of the azure types provider doesn't hint about what versions will be supported in the types package.

Example 2: The Kubernetes Provider

In the import statement currently used to load the Kubernetes types provider (import 'kubernetes@1.0.0') the version ('1.0.0') is a placeholder. The versioning of the types is reflected in the NuGet package reference version (from Bicep.Core.csproj), as shown below:

bicep/src/Bicep.Core/Bicep.Core.csproj

Line 44 in ac299a3

<PackageReference Include="Azure.Bicep.Types.K8s" Version="0.1.504" />

The actual definitions are generated are derived from a Kubernetes OpenApi spec (see here). These type definitions for Kubernetes are updated to v1.25.3 of Kubernetes and are not updated in between releases of Bicep.

These constraints customers to declare manifests that are compatible with Kubernetes clusters with versions v1.23 to v1.28 and there is no compatibility guarantee once Kubernetes versions evolve.

The customer has no hint about this potential problem since the version in the provider ('1.0.0') is a placeholder.

We propose that the Kubernetes provider is versioned using the versioning scheme of Kubernetes (e.g. '1.25.3', '1.28.0', etc...)

The types would match the OpenAPI specification for that release of Kubernetes and the underlying resource types would be versioned according to the Kubernetes API Server scheme (v1, v1beta1, v1alpha1, etc..). Its worth noting that such a version would also be OCI compliant, for example

mcr.microsoft.com/bicep/providers/kubernetes:1.25.3

@secure()
param kubeConfig string

import 'kubernetes@1.25.3' with {
  namespace: 'default'
  kubeConfig: kubeConfig
} as k8s

resource t 'flowcontrol.apiserver.k8s.io/FlowSchema@v1beta1' = {}

resource tt 'flowcontrol.apiserver.k8s.io/FlowSchema@v1beta2' = {}

Discussion on the Challenges

• Some users may want to always use the latest specification and are willing to take the risk of their deployment breaking. For example in the deployment to a staging environment that is not production.
• Some users may want to track a particular stable version of a package. For example with the kubernetes provider, maybe all the user wants is to have a resource type package tracking the latest PATCH version for a MAJOR.MINOR release. If we allow customers to specify OCI tags, then the user could gesture it in the following way:

import 'kubernetes@1.25' as k8s

[majastrz]

Regarding the Kubernetes part, I don't have an objection to matching the k8s versioning conventions. However, the 1.0.0 placeholder is due to the provider being baked into Bicep. Are we planning to switch that provider to pull it dynamically as well?

[asilverman]

Yes, I think we should eventually. Do you think we shouldn't pull it dynamically?

[anthony-c-martin]

This proposal does attempt to address the problem of being able to quickly understand which Az resource types are available in a given Az package, but I have the following concerns:

• In the long run, I feel that consistency is extremely important. If we imagine a world where we have 100s of providers available, each with their own versioning scheme and rules, I think the experience would be very confusing for users.
• This is a heavily Azure-centric solution - there's no guarantee the same discoverability benefit would apply to other providers.
• There are issues with practicality (for example, what happens if a provider misses publishing an API version 2022-01-01, and our 2022-01-01 package misses this type, but it is then available in 2022-02-01? - or vice versa with API deprecation).
• In practice, wouldn't people generally just want to be on "latest" for Azure? Is being able to be accurate with version targeting really as valuable as we think it is?
• The point Shenglong raised about being able to reason about breaking changes using semver.

In general, I'm not in favor of this proposal over using semver. Consistency is really my most important concern, but I also feel like we're over-optimizing on solving a particular (very Azure-centric) problem without fully understanding the details - for example, I would want to ask the question "Why do we feel we can't solve this problem with tooling or documentation?".

That said, I fully agree version/type discoverability is an important problem to solve, and appreciate the thought you've put into it here.

[asilverman]

• In the long run, I feel that consistency is extremely important. If we imagine a world where we have 100s of providers available, each with their own versioning scheme and rules, I think the experience would be very confusing for users.

I agree that consistency is important, however, its useful to consider that the syntactical enforcement of a string format doesn't strictly ensure consistency. That is, the 'semantic' factor of the SemVer standard is not enforceable strictly in the grammar. The syntax can be easily abused by a 3rd party provider to not follow the guidelines of SemVer (as is the case with many NuGet packages). For example,

• Artifacts in a OCI registry are not immutable, so an author can easily publish two different artifacts under the same tag (either by mistake or deliberately)
• The current syntax enforcement allows for a version 0.0.0-latest, 2020.20.12-preview and others.
• Domain specific concepts for resources may not fit well with the SemVer format, for example in the case of MS Graph where they are using the resource model of an API that is not semantically versioned. Or in the case of the hackathon GitHub provider which is tied to GitHub's API Versioning

With that said, I agree that for those packages that we author and make publicly available (i.e. Azure & Kubernetes) it's our responsibility (as package authors) to version the package sensibly in consideration with the domain specific concepts.

• This is a heavily Azure-centric solution - there's no guarantee the same discoverability benefit would apply to other providers.

To be clear I am not advocating for the sole use of only dates as a version, the proposal is asking to relax the syntactic enforcement of the SemVer format constraints so that each provider can choose what fits best for them.

As a direct consequence, the specifics for azure or kubernetes versioning schemes should be handled in their own separate topics that consider the domain specific requirements for each one of these distinct resources.

• There are issues with practicality (for example, what happens if a provider misses publishing an API version 2022-01-01, and our 2022-01-01 package misses this type, but it is then available in 2022-02-01? - or vice versa with API deprecation).

I am not sure how this could happen, isn't the version date for an API version set for the date the API is published? Maybe I am missing something here... I'm not an expert on the release process for new API versions.

With that said, I think that the topic of what the az version should be is out of scope for this proposal. Relaxing the syntactic enforcement of the semVer format doesn't preclude us from using it for az if its the one that makes best sense.

• In practice, wouldn't people generally just want to be on "latest" for Azure? Is being able to be accurate with version targeting really as valuable as we think it is?

Sounds like this is an argument for having a label like latest or stable be an acceptable version for az, but again, I think it merits a separate discussion.

I would want to ask the question "Why do we feel we can't solve this problem with tooling or documentation?"

The problem as stated in the description is that the language is too strict in the enforcement of a semVer format for the SpecificationString and the fact that this format is not always applicable and it can be easily abused. What do you mean by solving the problem with tooling or documentation?

[anthony-c-martin]

@asilverman I think probably easiest to discuss offline - I can talk you through my objections in more detail.

To give a simpler counterpoint:

Assumptions:

1. Each provider package MUST be immutable.
2. Providers can and will make mistakes or omissions when publishing API versions, and often go back to make corrections when issues are reported. This is VERY common.

The benefit of this proposal is that a user can purely use the version of the package to determine which resource types are supported. However, if we accept assumptions 1 & 2, this is not true - there will be cases where they need to arbitrarily increase the version, or snap to "latest" to deal with inaccuracies or omissions. At this point I really can't see any benefit over semver - the version is just an opaque string.

I am not sure how this could happen, isn't the version date for an API version set for the date the API is published? Maybe I am missing something here... I'm not an expert on the release process for new API versions.

I'll fill you in on this - background on the release process is crucial for this proposal.

[asilverman]

Each provider package MUST be immutable.

We have no way of guaranteeing this for packages that we don't author.

[jeskew]

I think it's still possible to identify breaking changes within resource bodies for the same API (e.g., by running a diff algorithm). However, I'm not sure if it's worth doing it, given that resource body breaking changes are extremely rare (actually, I don't know if such breaking changes would even be allowed to be accepted for merging today). In most instances, an azure-spec-rest-api PR updating an existing API version that is marked with "BreakingChangeReviewRequired" typically pertains to additions of optional properties, property annotations, and similar modifications that, in practice, do not constitute breaking changes for Bicep or ARM template users.

I agree that we should be able to automatically detect resource body breaking changes. Cribbing from @shenglol's suggestion, I would propose the following algorithm for detecting the type of change to make:

• PATCH version should be updated if:
  • Documentation strings are changed
• MINOR version should be bumped if:
  • New API versions are added*
  • New resource types are added*
  • New resource properties are added*
  • New az template functions are added
• MAJOR version should be bumped if:
  • Resource types or API versions are deleted*
  • az template functions are removed
  • There are breaking changes to az template functions

*: Can be detected with an automated CI pipeline

Semver requires that any new feature result in a minor version bump, and rolling new resource types or API versions in patch releases breaks this expectation. Following the above would mean that almost all updates would be a new minor version, with some not infrequent major version bumps when RP teams remove preview API versions. Patch versions would be theoretically possible but would never happen in practice.

That said, I would argue that semver is the wrong approach to take here because it introduces cognitive overhead without providing any of the affordances we see from semver with code dependencies. We have no plans to introduce features that would make semantic versioning useful, so tracking the semantic version of a provider increases complexity without increasing utility.

The main reason for this is that the current syntax requires that template authors pin a specific version of a provider. Semver matching notation (e.g., import 'az@^1.0.0') is not supported, meaning that reporting which releases contain breaking changes in the version number doesn't enable any additional automation. And given how often RPs retire preview API versions, I imagine we would train humans to ignore the major version number pretty quickly.

Secondly, there's no actual runtime effect of pinning a version and ignoring updates. Users could check their templates against an older version of the AZ provider, but they can't actually use it at runtime. Both RP contracts and az template functions will always use the latest version of the AZ provider (i.e., whatever the currently deployed state of ARM and ARM RPs might be). The primary consideration for a user is therefore, how close is my version of the provider to the current state of the world? Phrased different, how current is my version of the provider? You can't tell that from a semantic version, but you can tell it from a date.

I would argue that we should also support a special sentinel version value (e.g., import 'az@latest') so that users don't have to constantly tweak their templates in order to take advantage of separate vended provider types. This would be safe to do because users already have to pin their templates to specific API versions at the resource level, and that is how Azure RPs communicate which changes are incremental (performed within an API version) and which are breaking (performed by introducing a new API version). Using '@latest' is arguably safer than pinning a specific version of the provider because it reduces the drift between the Bicep compiler's view of the world and the state of the world that will be enforced when the template is deployed.

For non-Az providers, is the plan to have multiple versions of the server side code running? If so, semver would make sense, as the import communicated in the template would be respected at deploy time. This capability seems particularly important for providers that talk to different instances of something (e.g., k8s), where each instance may be running a different version of the thing ARM extensibility is talking to. It seems less important for providers where ARM will communicate with a centralized control plane (e.g., az or graph), where deployment requests will always be handled by the latest version of the API.

[asilverman]

Although this comment is specific to the az provider, I think its adding to the conversation in the sense that it reflects that the enforcement of the semVer format is not something the language MUST enforce. If we relax the format to a valid OCI Tag (128 ASCII alphanum chars) we can still debate what the version for az will be and choose semVer or Date or something else. The inverse is not true.

[shenglol]

After numerous rounds of internal discussions, we have arrived at the decision that neither semantic versioning nor the use of dates as version identifiers will fully address our requirements. Instead, we have chosen to adopt the basic semantic versioning format, denoted as X.Y.Z-suffix. However, in response to the distinct needs of Azure type management, which involves safely publishing changes for both Azure resource types and ARM template functions, we are introducing a custom versioning schema, as outlined below.

A versioning schema includes rules and conventions for incrementing version numbers.

Normal version

A normal version number MUST be in the format of X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes. X is the primary version, Y is the secondary version, and Z is the non-breaking change version.

Version number X

The primary version number X signifies significant alterations to ARM template functions that may result in breaking changes. It MUST be increased by 1 in the following scenarios:

• Any of the existing ARM template functions undergoes modifications in its signature or behavior.
• The removal of any existing ARM template function. However, the removed function MUST remain available in older versions to ensure backward compatibility.

Version number Y

The secondary version number, represented by Y, addresses breaking changes related to Azure resource types. It MUST be incremented by 1 when:

• Any of the existing resource types undergoes modifications that breaks the existing contract, including
  • The removal of an existing property
  • The addition of a required property
  • Changing the type of an existing property
  • Narrowing the range of the possible values of an existing property
  • Etc.
• Any of the existing resource types is removed.

The secondary version number MUST be reset to 0 when the primary version number is incremented.

Version number Z

The non-breaking change version number, denoted by Z, is designed to account for minor or patch modifications to ARM template functions OR Azure resource types that do not introduce breaking changes. It should be incremented by 1 under the following circumstances:

• A new ARM template function is added.
• An overload of an existing ARM template function is added.
• A new Azure resource type is added.
• A new API version of an existing resource type is introduced.
• Changes made to documentation strings for ARM template functions or Azure resource types.

The version Z number MUST be reset to 0 when either the primary OR secondary version number is incremented.

Pre-release version

A pre-release version MAY be denoted by appending a hyphen and an identifier immediately following the patch version, for example, 1.0.0-alpha, 1.0.0-preview. Pre-release versions have lower precedence than the associated normal version. A pre-release version indicates that the version is unstable and may not meet the intended compatibility requirements specified by its associated normal version. It's important to note that a pre-release version MAY be deleted. Since the Bicep team does not control the publication of Azure resource types, pre-release versions will typically be used to test ARM template function changes.

Initial version

To establishes a clear starting point for the versioning of the Azure type package, the version for the initial release of the Azure type package SHOULD be 0.0.1.

Appendix: alternative solutions considered

X.Y.DATE

Using X.Y to capture breaking and non-breaking changes to ARM template functions and appending DATE to signify the date when a type package is generated for Azure resource type changes, for example, 1.0.20231001 or 1.1.20231101. However, we decided against this schema primarily because it could create incorrect expectations. An Azure type package version like 1.0.20231001 may not necessarily contain all resource type API versions older than 2023-10-01. Customers using this Azure resource type version might wonder "why is storage account version 2023-08-01 not available in 1.0.20231001 without realizing that the Swagger definition for storage account version2023-08-01 was published on 2023-10-02. Additionally, relying solely on a date does not adequately inform customers about the nature of changes made to the resource types, making it less transparent and less informative for users.

Aligning Azure type package version with Bicep version

This approach aligns the Azure type package with the same major and minor version numbers as Bicep and calculates its patch version number. While it simplifies version management for the Azure type package, thus saving efforts for the Bicep team, it fails to provide a clear correlation between version numbers and changes to ARM template functions and Azure resource types. This lack of clarity diminishes its value to customers who rely on versioning numbers to understand the scope and impact of updates.

References

• Semantic Versioning 2.0.0