Skip to content
This repository has been archived by the owner on Mar 11, 2021. It is now read-only.

Reconsider how Azure API versions are exposed in the programming model #169

Closed
joeduffy opened this issue Dec 11, 2020 · 9 comments · Fixed by pulumi/pulumi-azure-native#481
Assignees
Labels
1.0 Track for 1.0 release impact/usability
Milestone

Comments

@joeduffy
Copy link
Member

joeduffy commented Dec 11, 2020

I am finding working with the Azure NextGen provider to be a bit clumsy in comparison to our usual patterns, due to the way we expose API versions. This is simply my own feedback, and I'm curious whether others agree/disagree.

The way API versions are front and center feels very "web API", and specific to the way Azure versions their APIs, and not something I'd expect so prominently in a library's surface area. Also, I can't imagine most end users would want to peg to a specific version, versus just using latest, so forcing users to think about and choose a "version" so early in developing code against these libraries just feels odd -- indeed, for me, it was jarring. I mean, why wouldn't I just pick latest?

It also seems odd to me that versions are per-service rather than just being global to the entire package.

An alternative way to structure this would be:

  • Expose latest resources automatically from the various modules -- so, for instance, new azure.resources.ResourceGroup(..) would pick up the latest API by default.

  • Provide a way to import the API definitions for alternative versions if you want, at a package-wide level -- so, for example, import * as azure20190501 from "@pulumi/azure-nextgen/v20190501".

This feels more intuitive to me. As an added bonus, it should perform better because we wouldn't parse all the different API version variants when you imported the default version alone. This could have language-specific implications, however.

@mikhailshilkov
Copy link
Member

Overall, my hypothesis is that this is much more natural for folks familiar with ARM. Our raw exposal of ARM API sort-of requires good understanding of ARM overall, there's no shortcut, unfortunately.

I mean, why wouldn't I just pick latest?

  1. You may get breaking changes (code-level and behavior-level) every time you update to a new version of the provider (including minor versions).
  2. The latest may not work yet (e.g. Not possible to import app service plan #166)
  3. It may not have all the features (e.g., SQL and Authorization have old stable latest but new features in preview only)
  4. Your ARM templates won't map to the latest 1:1.

As mentioned in #167, the plan is actually to discourage using latest for long-existing stacks.

It also seems odd to me that versions are per-service rather than just being global to the entire package.

You would get ~50-100 versions to choose from - how would you find the one you need? It's easier to start your selection with the resource provider and get a limited set of versions.

Expose latest resources automatically from the various modules -- so, for instance, new azure.resources.ResourceGroup(..) would pick up the latest API by default.

We discussed this and can still do this, but that doesn't really match the advice not to use it for production stacks.

Provide a way to import the API definitions for alternative versions if you want, at a package-wide level -- so, for example, import * as azure20190501 from "@pulumi/azure-nextgen/v20190501".

azure20190501 doesn't make much sense to me. But you can do import * as resources20200901 from "@pulumi/azure-nextgen/resources/v20190501 today.

As an added bonus, it should perform better because we wouldn't parse all the different API version variants when you imported the default version alone.

Indeed. All our manually-created examples and templates don't use blank imports for this reason. Instead, they import targeted separate modules. Our docs generation can't do that yet.

Curious what others think @lukehoban @leezen @infin8x

@joeduffy
Copy link
Member Author

All makes sense, and I suspect you're 100% right here. Just take this as feedback from a relative Azure newbie who just wanted to get up and running, and spelunk a bit through the Azure services, and wished it was simpler to navigate and understand how to do the right thing. The versioning of the package and the APIs within that package is definitely a strange, non-standard pattern, and yet I understand from your reply why it ended up that way.

@lukehoban
Copy link
Member

JFWIW - I personally expected we would get a fair bit of similar feedback during the preview, and would find ourselves wanting to promote the latest submodule to just being available at top level to make the "onramp" simpler, and the simplest code examples simpler (avoid all the latest and aliasing).

I do believe the "right" thing for heavy users will be to learn about API Versions and pin to them, as there are real versioning pains they will feel otherwise. But luckily, that pinning is something they can do at the time they might be impacted by a breaking change, which means it is less critical they buy into this complexity right away. That is, if they start with just the simple default, then if/when they encounter a breaking change, they have the option to pin the API Version.

There are a few things we could do to make this a better overall user experience if we did want to go this direction:

  1. Make sure the APIs emitted to the root include in docs what API Version they are (currently) from, so that this is discoverable, and so that changing to the strongly-versioned form is easier. We've heard this pain point several times from users that this is difficult to discover today.
  2. Ensure that changing the API Version - even if it happens implicitly as part of using one fo the latest (or root) resources that has had a change to it's default version and thus a potential breaking change - show up as diffs in the pulumi preview - so that users understand that something material is changing when they do upgrade to a new version of the @pulumi/azure-nextgen package.

The latest may not work yet (e.g. #166)

Does the Azure SDK have this same problem? I still am not exactly sure how our latest is different from Azure SDK notion of latest.

@mikhailshilkov
Copy link
Member

Does the Azure SDK have this same problem?

I guess they don't. As far as I can tell, they don't have a fully automated process to release a new SDK version as soon as API definitions change and they do have some hand-holding to avoid these situations. One part of that are md files like here which instruct each SDK which version to use by default (lines like tag: package-2020-09). Although, I've seen these files (at least, the main ones) updated to versions that don't work for me (1, 2).

Often, an API version would change with a comment "SDKs don't use it yet".

@mikhailshilkov mikhailshilkov added 1.0 Track for 1.0 release impact/usability labels Dec 13, 2020
@MisinformedDNA
Copy link

Overall, my hypothesis is that this is much more natural for folks familiar with ARM.

As someone who has worked on ARM for some years, I can tell you that, for me, I take zero thought of the API version until I have to. Usually, I just use whatever version was there when I copied the template from elsewhere. I would like to take zero thought of the API version in Pulumi as well.

As mentioned in #167, the plan is actually to discourage using latest for long-existing stacks.

I disagree with this plan. It might work better based on Pulumi's current versioning scheme, but it is contrary to the way developers work with packages in real languages. This would also make shared libraries untenable because now library developers either have to recode every thing for each version of each resource or they just pick which version has the most users (probably latest).

I mean, why wouldn't I just pick latest?

  1. You may get breaking changes (code-level and behavior-level) every time you update to a new version of the provider (including minor versions).
  2. The latest may not work yet (e.g. Not possible to import app service plan #166)
  3. It may not have all the features (e.g., SQL and Authorization have old stable latest but new features in preview only)
  4. Your ARM templates won't map to the latest 1:1.

Here is how I would fix these issues and make it easier for those who want to work with "latest" or just newbs:

  1. You may get breaking changes (code-level and behavior-level) every time you update to a new version of the provider (including minor versions)

Yes, because of the way Pulumi is doing this. Currently, the package version can change from X.3 to X.4 and, even though it may seem like a harmless update, it can actually introduce breaking changes where most developers wouldn't expect one.

Example:

Just spent the last few hours trying to work out why my Pulumi config wasn't working for a project when it is literally a copy paste of a previous project that does work. Turns out this docker image was pulling 2.12.2 of Pulumi, whereas the previous project was on 2.11.2. Looks like 2.12.x introduced breaking changes AWS around certificate generation and domain validation.

This is easily solved by updating the major version number every time a new Azure API version is released.

  1. The latest may not work yet (e.g. Not possible to import app service plan #166)

Again, this issue becomes much less critical when the major version number changes. This is because developers understand that a new major version implies potential breaking changes. The next section also helps.

  1. It may not have all the features (e.g., SQL and Authorization have old stable latest but new features in preview only)

For this (and many other good reasons), we need to break up the areas and versions into separate packages. Currently, we have one package that includes everything:

  • Pulumi.AzureNextGen (latest and old versions)

I propose the following package dependency structure:

  • Pulumi.AzureNextGen.All (contains latest and versioned packages; valuable to ARM2Pulumi)
    • Pulumi.AzureNextGen (contains latest of all areas)
      • Pulumi.AzureNextGen.Storage (Latest namespace is removed)
      • Pulumi.AzureNextGen.Web (Latest namespace is removed)
      • ...
    • Pulumi.AzureNextGen.Storage.V20190601
    • Pulumi.AzureNextGen.Storage.V20200801Preview
    • Pulumi.AzureNextGen.Web.V20180201
    • Pulumi.AzureNextGen.Web.V20200601
    • ...

"So, mostly, everything is the same, just multiple packages" But sooo much more flexible and useable.

  1. Your ARM templates won't map to the latest 1:1.

That's fine. ARM2Pulumi can still use the versioned resources.

Results

  • Removing .Latest makes it much easier to migrate from Pulumi.Azure
  • Users who just want to use the latest can easily do so
  • Users are not blindly updated to a new version
  • Users who want to pin versions can easily do so
  • Users are not forced to upgrade
  • Old version-named packages do not need to be republished every time
  • arm2pulumi can continue as-is without any significant change

@joeduffy I was in the same boat as you. At first, I was like WTH!?! Then @mikhailshilkov explained it to me and it all seemed to make sense. But, now, as someone who has spent a lot of time with this provider, it just doesn't work well. As @lukehoban explained, "latest" should be the default and the versioned packages are just for fallback.

My original Slack thread

@mikhailshilkov
Copy link
Member

Here is our thinking for API versioning that we intend to have at 1.0 of this provider.

We have two major goals here:

  1. Expose the full surface of Azure API, with new updates continuously coming to the Pulumi SDKs
  2. Provide great user experience for common usage scenarios, i.e.
  • Familiar - similar to the "classic" Azure provider
  • No need to think about API version up-front
  • No unneeded breaking changes within a major version

We will solve these two goals separately:

  1. We will keep publishing all API versions for all resource providers in separate modules/namespaces. Nothing changes here.
  2. We will provide top-level resources, each using a stable (but potentially not bleeding-edge latest) API version.

How the top-level resources work:

  1. We will pick a version for every Azure resource and place it to the top module of each resource provider. E.g., you will be able to create a storage account like
new Pulumi.AzureNextGen.Storage.StorageAccount("sa", ...);
// or
new azure_nextgen.storage.StorageAccount("sa", ...);
  1. Resources that have preview versions but no stable versions are also available in the top module (they were not in Latest).

  2. At 1.0, for a given resource, we will likely pick an API version that is the latest. If a later version is published by Microsoft and it contains breaking changes, we will NOT introduce this version to the top-level resource. It will be available under a versioned namespace only.

  3. At 2.0, whenever it comes, we will bring all those postponed breaking changes by promoting top-level resources to their respective latest API versions.

  4. We will encourage everyone to use top-level modules unless they have a reason not to do so (e.g., they need a newer version with its new features).

  5. All "Latest" modules/namespaces become deprecated and will be removed before 1.0. You will be able to migrate to top-level resources without re-creating Azure resources.


We will explore the potential evolution of our SDKs that may allow us to make the top-level resources "multi-versioned". Any results will come after the 1.0 release.

I encourage everyone to 👍 this comment if it makes sense to you, or leave a comment below with any concerns.
Thank you!

@MisinformedDNA
Copy link

I'd like to see the top-level modules as a separate NuGet package since

a) It's opinionated
b) It won't be updated at the same rate as the full API surface
c) It greatly improves Intellisense in VS (can't comment for other languages)

Thoughts?

@mikhailshilkov
Copy link
Member

@lukehoban I think you had a strong opinion against splitting the packages. Could you respond to @MisinformedDNA?

@MisinformedDNA
Copy link

And this:

image

VS only recommends three possible classes, but since there are so many classes named the same, it, more often than not, recommends all the ones you don't want. If there were fewer classes (slimmer packages), this issue would come up far less often.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
1.0 Track for 1.0 release impact/usability
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants