Complete the SemanticVersion work

[HemantMahawar]

• Support SemVer in #Requires (PowerShell version (-Version) as well as module version (-Modules))
• Allows cmdlet parameters to consume SemVer instead of System.Version (argument transformation maybe?)
  • Users can convert Semver to System.Version using that attribute before passing to the cmdlet parameter
  • Cmdlet author can mark the attribute on the parameter and users can pass Semver string

=======================
User Experience:

• On disk, module versions presents - 1.0.0, 1.0.0-alpha17, 1.0.1

• Result:

  Command	Expected Output	Comments
  Import-Module -Version 1.0.2	Error	Version is alias to min version
  Import-Module -MinimumVersion 1.0.0	Module with version 1.0.1 should be loaded
  Import-Module -Version 1.0.0-alpha15	Module with version 1.0.1 should be loaded
  Import-Module -Name	Module with version 1.0.1 should be loaded	Latest version of module should be loaded
  Import-Module -RequiredVersion 1.0.0	Module with version 1.0.0 should be loaded

Note that 1.0.0 is higher than 1.0.0-alpha17, so if 1.0.1 doesn't exist, then Import-Module would load 1.0.0.

[SteveL-MSFT]

We should wait for corefx support of semver (https://github.com/dotnet/corefx/issues/13526) rather than continue to have our implementation

[JamesWTruher]

The WG discussed this and in the absence of underlying support from the .NET team, we don't believe we should undertake any effort in this area.

[ThomasNieto]

@JamesWTruher if that is the case please remove the SemanticVersion support from module versions as the half support we have today hurts the PowerShell experience. My opinion is that you fully support it or remove it from PowerShell and PSResourceGet.

[ThomasNieto]

Brought this up with the following points in the PS community call.

• The WG decision does not have any context, address any issues, or consider any alternate solutions.
  As such indicates Semantic Versioning is not supported in PowerShell and by extension PSResourceGet and should be removed

• The scope of work is allowing Semantic versioning in module cmdlets
  which could be accomplished by adding a (65KB) dependency on NuGet.Versioning

• Here are some of the issues when PSResourceGet supports Semantic Versioning while the engine does not
  Issue 1# - Prerelease binary modules cannot be updated to stable.
  The largest impact of this is PSReadLine as users who install the prerelease version will have to under go workarounds to install the stable version; thus hurting testing adoption

  Issue 2# - Prerelease and stable versions cannot be installed at the same time

  Issue 3# - Formatter displays a fake Prerelease property causing user confusion

• PowerShell should either fully support Semantic versioning in the engine or remove it completely and in PSResourceGet

• My concern about WG verdicts and lack of transparency could be addressed by
  WG meetings being recorded, having meeting minutes, and a publicly accessible vote with accompanying concurring or dissenting opinions.

  This would help users know their issue has been earnestly considered

• I want the PowerShell committee to review the future of Semantic Versioning as PSResourceGet should not support a versioning scheme that PowerShell cannot

[ThomasNieto]

@SydneyhSmith please reopen this issue as was discussed on the community call.

[microsoft-github-policy-service]

This issue has been marked as declined and has not had any activity for 1 day. It has been closed for housekeeping purposes.

[microsoft-github-policy-service]

📣 Hey @HemantMahawar, how did we do? We would love to hear your feedback with the link below! 🗣️

🔗 https://aka.ms/PSRepoFeedback

Microsoft Forms

[ThomasNieto]

@SydneyhSmith the issue closed again because the tags are still on it.

[SeeminglyScience]

• The scope of work is allowing Semantic versioning in module cmdlets
  which could be accomplished by adding a (65KB) dependency on NuGet.Versioning

This was discussed, but adding a dependency can cause more issues than just the size. Newtonsoft.Json for example is a dependency we took that is a large pain point for a lot of folks.

That doesn't mean we can't add dependencies, it just raises the bar for impact significantly.

• Here are some of the issues when PSResourceGet supports Semantic Versioning while the engine does not
  Issue 1# - Prerelease binary modules cannot be updated to stable.
  The largest impact of this is PSReadLine as users who install the prerelease version will have to under go workarounds to install the stable version; thus hurting testing adoption
  Issue 2# - Prerelease and stable versions cannot be installed at the same time

PSReadLine now increments the Revision (or maybe Patch?) number for the version when it releases a new preview or GA. It's not perfect, but it is a workaround that works well.

That said, keep in mind this issue is very broad. The WG is only saying that we cannot commit to fixing every issue with the semver type and also everything around it - at least without support from the BCL. For example, I do personally think it would be smart to at least change module import look up to look for semver folders, even if PSRG won't be able to save them to that format for likely a few years. Still should get the ball rolling on it, I've opened #21371 to track that.

Issue 3# - Formatter displays a fake Prerelease property causing user confusion

FWIW I very much dislike fake properties in formatting in general, and I agree that at the very least an ETS property should be added. Feel free to open an issue for that if you'd like

• My concern about WG verdicts and lack of transparency could be addressed by
  WG meetings being recorded, having meeting minutes, and a publicly accessible vote with accompanying concurring or dissenting opinions.
  This would help users know their issue has been earnestly considered

I don't disagree, but it is also a bit of a give and take there. The more public it is, the harder it is to have a candid discussion - which means it takes longer and we get to even less issues than we do now.

It's the same reason why we often can't explain in great detail everything that was discussed, because writing all of that up takes a lot of time.

It's definitely something we're thinking about how to improve.

[hilari0n]

• The scope of work is allowing Semantic versioning in module cmdlets
  which could be accomplished by adding a (65KB) dependency on NuGet.Versioning

This was discussed, but adding a dependency can cause more issues than just the size. Newtonsoft.Json for example is a dependency we took that is a large pain point for a lot of folks.

That doesn't mean we can't add dependencies, it just raises the bar for impact significantly.

Uhm... So one of the reasons of not using NuGet.Versioning is that it would be a new/added reference.
From what I see, NuGet.Versioning is already delivered with Microsoft.PowerShell.PSResourceGet (at least since version 1.0.0), which is delivered with PowerShell.

And - as far as I know - it's quite improbable for it to go away, as it's one of the core mechanics of NuGet packages versioning, which are something, that PowerShell relies on (and should probably start embracing those Nuget.* packages, instead of building its own mechanics for NuGet handling).
And PowerShell is apparently based on .NET SDK (i.e., PowerShell release notes always reference a .NET SDK version, not e.g., .NET runtime version, quote from release notes 7.4.1 "Bump .NET 8 to version 8.0.101. Update .NET SDK and dependencies for v7.4.1"), which does contain NuGet.Versioning.

[SeeminglyScience]

Uhm... So one of the reasons of not using NuGet.Versioning is that it would be a new/added reference. From what I see, NuGet.Versioning is already delivered with Microsoft.PowerShell.PSResourceGet (at least since version 1.0.0), which is delivered with PowerShell.

True! But it does so in its own assembly load context which won't conflict. For PowerShell to fully embrace the type it would need to be registered and loaded at start up and in the default ALC. Even if we were able to lazily load it somehow (which we likely wouldn't be able to do due to it needing to be referenceable from script) we would still need to lock down in our assembly resolver which assembly is loaded.

[SeeminglyScience]

And PowerShell is apparently based on .NET SDK (i.e., PowerShell release notes always reference a .NET SDK version, not e.g., .NET runtime version, quote from release notes 7.4.1 "Bump .NET 8 to version 8.0.101. Update .NET SDK and dependencies for v7.4.1"), which does contain NuGet.Versioning.

Bumping the SDK is required to get the latest runtime. NuGet.Versioning is not shipped with every .NET project, it's just a compile time dependency

[hilari0n]

And PowerShell is apparently based on .NET SDK (i.e., PowerShell release notes always reference a .NET SDK version, not e.g., .NET runtime version, quote from release notes 7.4.1 "Bump .NET 8 to version 8.0.101. Update .NET SDK and dependencies for v7.4.1"), which does contain NuGet.Versioning.

Bumping the SDK is required to get the latest runtime. NuGet.Versioning is not shipped with every dotnet project, it's just a compile time dependency

SDK-s are not required to get the latest runtime. Runtime is also distributed separately and you can use it that way.
Also, what latest runtime you would get from SDK 8.0.203, over the one from SDK 8.0.202? You can even use SDK 9.0 to build an application for the 8.0 runtime and it will work with that runtime (just as I'm using SDK 8.0 to build .NET apps for runtimes 6.0 and 7.0 along with ones for 8.0).
SDK is a compile-time dependency in a very similar way, as NuGet.Versioning is.

It's baffling, that I'd get the exact SDK version number used for building and packaging in the release notes, but the actual runtime version, which is packaged with PowerShell, is something I have to look up. It's also baffling, that PowerShell needs to use the self-contained model and deliver the runtime, so I have to have another copy of it on my machine, (and wait for separate security updates for PowerShell, when there are runtime security updates available).

Going back to having the NuGet.Versioning NuGet in PowerShell:
It's a much less breaking change to have it there and even have it fixed to some version (and have potential issues with modules wanting to rely on a different one), than changes like e.g., dropping the NJsonSchema NuGet in 7.4.0 completely (and breaking a lot of uses of Test-Json). If someone's OK with messing up how people work win Json in PowerShell, when switching from 7.3 to 7.4, but is not OK with adding NuGet.Versioning somewhere between Windows PowerShell 5.1 and PowerShell 7.4, then I can't grasp the thought process behind it.

[SeeminglyScience]

SDK-s are not required to get the latest runtime. Runtime is also distributed separately and you can use it that way.

The SDK is what we use to build PowerShell. If you build a self contained project with a lower version of the SDK then you get a lower version of the runtime.

Also, what latest runtime you would get from SDK 8.0.203, over the one from SDK 8.0.202?

Sorry, I don't know the specifics of every SDK version. I imagine that some of them are only tooling changes, but for example from 8.0.201 to 8.0.203 the runtime version is higher when compiling a self contained project.

You can even use SDK 9.0 to build an application for the 8.0 runtime and it will work with that runtime (just as I'm using SDK 8.0 to build .NET apps for runtimes 6.0 and 7.0 along with ones for 8.0).

Yeah for sure, you just can't use an older version of the SDK to build for a newer version.

SDK is a compile-time dependency in a very similar way, as NuGet.Versioning is.

Yep! Exactly

Going back to having the NuGet.Versioning NuGet in PowerShell: It's a much less breaking change to have it there and even have it fixed to some version (and have potential issues with modules wanting to rely on a different one), than changes like e.g., dropping the NJsonSchema NuGet in 7.4.0 completely (and breaking a lot of uses of Test-Json). If someone's OK with messing up how people work win Json in PowerShell, when switching from 7.3 to 7.4, but is not OK with adding NuGet.Versioning somewhere between Windows PowerShell 5.1 and PowerShell 7.4, then I can't grasp the thought process behind it.

I wasn't involved with that change and don't know much about it, but even if breaks were intentional there - that doesn't really impact the decision here. One break doesn't mean all breaks are warranted. Especially if a previous change had a bad outcome, if anything that would be an example of why not to do it, not why to do it more.

[daxian-dbw]

It's baffling, that I'd get the exact SDK version number used for building and packaging in the release notes, but the actual runtime version, which is packaged with PowerShell, is something I have to look up.

This can be improved to link to the .NET release page of that version of .NET SDK. It will make it easy for you to find out what version of .NET runtime coming with it.

It's also baffling, that PowerShell needs to use the self-contained model and deliver the runtime, so I have to have another copy of it on my machine

Then maybe you want to use the PowerShell global tool instead, which depends on the corresponding runtime to be already installed separately on your machine.

dropping the NJsonSchema NuGet in 7.4.0 completely (and breaking a lot of uses of Test-Json)

Although that breaking change is not relevant to this discussion, I want to add some context about it:

1. This comment Supporting later versions of JSON Schema #18009 (comment) was why we decided to accept that change.
2. We considered it a breaking change at the time in the sense that we were switching to a different library, but we didn't expect the behavior of Test-Json to be changed.

[SteveL-MSFT]

Keeping this open as the .NET issue is still open. If they decide to never do it, we can revisit, but it doesn't make sense for PS7 to have it's own semver implementation with the risk of .NET having a different one and the two may not be compatible.

[ThomasNieto]

@SteveL-MSFT can you get into contact with the .NET team to see if it is ever going to be on their roadmap because if history is any indication they have no intention of ever doing it as 8 years has already gone past since this issue has been opened. What is the timeframe PowerShell will wait before doing its own implementation because we're at 8 years now, a decade? The PS Committee should have a reasonable drop dead date if .NET will not include the type then PS will do the implementation. We cannot have this issue open for another 8 years.

Here is the deal, the PowerShell team decided to implement semantic versioning in PowerShellGet/PSResourceGet, PowerShell by extension should as well due to various issues laid out above. If PowerShell does not want to support semantic versioning then it should be removed from PSResourceGet. A firm decision needs to be made by the WG/Committee in which direction they would like to go in and in a reasonable timeframe.

[SteveL-MSFT]

The main problem that might not be completely apparent is that if PS7 has a implementation of semver, then we need to support it indefinitely as the core platform. Individual modules may do what they need for their cases, but other apps not specific to that domain would not take a dependency on it. Although not ideal, we can continue to not have a semver type.

[ThomasNieto]

At this point I just want the module issues relating to semver prerelease modules fixed as that is the main issue.

From the discussion on this thread these are the non-starters:

• No competing semver types from PS and .NET
• No use of [S.M.A.SemanticVersion] as it violates competing types
• No new library dependencies like [NuGet.Versioning] as version conflicts can/will happen and can't use ALC.
• No breaking changes such as changing property types or parameter types

Here is my proposal address fixing semver prerelease modules without breaking those non-starters.

1. Complete Import-Module should be able to find modules with semver versioned folders #21371
2. Create a new parameter called VersionRange that accepts a String. This parameter would accept NuGet version range syntax like PSRG. This is needed as the existing version parameters are type [Version] and it would be a breaking change to change types. Another benefit is to align PSRG and PS into a common interface simplifying the number of parameters needed. The last and main benefit to this is that the parsing of the string can be handled internally not exposing a semver type to the user while handling both [System.Version] and semver.
3. Optional, move MinimumVersion, MaximumVersion, RequiredVersion into a legacy parameter set (might not be able to with the amount of parameter sets today) or hide them for backwards compatibility
4. Add a SemanticVersionString property on PSModuleInfo with type of String? that is populated when the version can be parsed into a semver. This is to not expose a semver type to the user
5. Update PSRG to use semver prerelease folders when on PS Versions supporting semver prerelease. This should not be an issue as if the user requested the prerelease version on an older PS version would be the current state. If the new PS version is used then it would be in the prerelease folder. Then if the user requested the stable version it would not run into the binary update issue and be placed into the correct version folder. This is not a breaking change as the user requests the version if they install the prerelease or stable versions.

Then once a .NET type is released.

1. Take dependency on semver .NET type
2. Update internal logic to use semver .NET type
3. Add SemanticVersion property using semver .NET type on PSModuleInfo

This proposal solves the main issues with the current state while not breaking any of the non-starters and leaving room in the future for a real .NET semver type.