[WebToolsE2E] With VS 17.10 P1 installed (including Aspire 8.0 P3), if we then install .NET SDK 9.0 P1, VS does not show the Aspire templates anymore

[balachir]

REPRO STEPS

1. Clean machine: Win11 x64 23h2 ENU
2. Install VS 17.10 Preview 1 (includes .NET SDK 8.0.200 and .NET Aspire workload 8.0 Preview 3)
3. Open VS, create new solution using 'Aspire Starter Application' template, build and run using VS -- it works fine
4. Close VS
5. Install .NET SDK 9.0 Preview 1
6. Open VS

ACTUAL
During project creation,

• The 'Aspire Starter Application' and other aspire templates don't show up in VS New project dialog anymore
  
• When creating a new web app or api app, the 'Enlist in Aspire' checkbox does not show up in New Project dialog
  

If you go to command-line and run dotnet --info, it incorrectly shows that the 9.0 Aspire workload is installed, even though I didn't run any commands to install the 9.0 Aspire workload.

EXPECTED
During project creation,

• VS should continue to show the 8.0 Aspire templates
• When creating a new web app or api app, the 'Enlist in Aspire' checkbox should show up in New Project dialog

[phenning]

More information, after installing NET 9.0

The template locator reports no template packages:

Finding sdk version for 17.10
sdk version: 9.0.100-preview.1.24101.2, dotnet root path: C:\Program Files\dotnet			
template packages: 

Although the workload IS installed:

installed workloads: { ... snip ... "microsoft.net.sdk.aspire": "9.0.0-preview.1.24079.4" }

dotnet --info:
.NET SDK:
 Version:           9.0.100-preview.1.24101.2
 Commit:            6bbd460f4d
 Workload version:  9.0.100-manifests.c840f51f

.NET workloads installed:
 [aspire]
   Installation Source: VS 17.10.34607.79, VS 17.10.34614.196
   Manifest Version:    9.0.0-preview.1.24079.4/9.0.100-preview.1
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\9.0.100-preview.1\microsoft.net.sdk.aspire\9.0.0-preview.1.24079.4\WorkloadManifest.json
   Install Type:              Msi

C:\>dotnet workload list

Installed Workload Id      Manifest Version                               Installation Source
---------------------------------------------------------------------------------------------------------------
aspire                     9.0.0-preview.1.24079.4/9.0.100-preview.1      VS 17.10.34607.79, VS 17.10.34614.196

sdk-manifests\9.0.100-preview.1\microsoft.net.sdk.aspire\9.0.0-preview.1.24079.4\WorkloadManifest.json:

    "Aspire.ProjectTemplates": {
      "kind": "template",
      "version": "9.0.0-preview.1.24079.4"
    },

However, no template pack is installed for 9.0:

C:\Program Files\dotnet\template-packs>dir
 Volume in drive C is Windows
 Volume Serial Number is 7273-32B5

 Directory of C:\Program Files\dotnet\template-packs

02/14/2024  02:18 PM    <DIR>          .
02/15/2024  03:27 PM    <DIR>          ..
10/25/2023  11:25 AM           180,975 Aspire.ProjectTemplates.8.0.0-preview.2.23525.3.nupkg
12/19/2023  08:05 PM           195,034 Aspire.ProjectTemplates.8.0.0-preview.2.23619.3.nupkg
02/06/2024  03:27 AM           246,363 Aspire.ProjectTemplates.8.0.0-preview.3.24105.21.nupkg
01/25/2024  09:42 PM           125,214 Microsoft.Android.Templates.34.0.79.nupkg
01/30/2024  08:56 AM           187,774 Microsoft.iOS.Templates.17.2.8022.nupkg
01/30/2024  08:56 AM           113,018 Microsoft.MacCatalyst.Templates.17.2.8022.nupkg
11/02/2023  08:44 PM           609,363 Microsoft.Maui.Templates.net7.7.0.101.nupkg
01/15/2024  07:16 PM           596,645 Microsoft.Maui.Templates.net8.8.0.6.nupkg
               8 File(s)      2,254,386 bytes

After running dotnet workload install aspire:
Updates build to preview.1.24113.3, and template packs are installed.

Aspire now shows up for net 9.0 - net 8.0 template is totally gone.
template packages: C:\Program Files\dotnet\template-packs\aspire.projecttemplates.9.0.0-preview.1.24113.3.nupkg

I believe that needs to happen, is that when 9.0 SDK is installed, the Template Locator must still return the NET 8 templates, so that they can be used by Visual Studio if we expect customers to have both NET 8 and NET 9 SDKs SxS and able to support both 8.0 and 9.0 Aspire on same Dev box. Alternatively, the NET 9 aspire templates could target ALL supported, released TFMs like Winforms do (https://github.com/dotnet/winforms/blob/980b02212d847cc3e9f294bbb900ccb3249bc0a2/pkg/Microsoft.Dotnet.WinForms.ProjectTemplates/content/WinFormsApplication-CSharp/.template.config/template.json#L27)

@marcpopMSFT, FYI related to template locator and SxS issues in templates

[phenning]

@joperezr FYI, as well.

[joperezr]

Thanks @phenning, we were actually talking about this problem yesterday in Tactics with @marcpopMSFT. Is there a way to "force" the template location logic to use a specific SDK band? I know that if you already have a project, you can use a global.json that specifies 8.0.200 for example in order for the 9.0.100 one to not get used, but when it comes to brand new projects, there isn't a global.json that can be used.

Is there some sort of environment variable to be used as a workaround for this? If not, would it be possible to add it? I understand using environment variables is not a very user friendly solution, but IMO it is better than the current alternative which is to uninstall 9.0 SDK if you want 8.0 aspire templates to come back.

[phenning]

@joperezr I just tested something like that out

Currently we do something like this:

sdkVersion = Locator.GetSdkVersionUsedInVS
packs = Locator.GetTemplatePackages(sdkVersion)

i've updated it to (essentially)

sdkVersion = Environment.GetEnvironmentVariable("OVERRIDE_VARNAME_TBD") ?? Locator.GetSdkVersionUsedInVS
packs = Locator.GetTemplatePackages(sdkVersion)

SET OVERRIDE_VARNAME_TBD=8.0.300-preview.24101.10

And box lights up for net8 and doesn't show for 9.0

I think this will be a good interim workaround in 17.10 preview 2 and later until we solve the larger issue. I'll probably add some code to make sure that the override is actually still present as well to guard against stale environments.

[marcpopMSFT]

Sounds like @phenning was able to add a way to force a specific version in NPD. Phil, this is different than using global.json though correct as it's only getting the list of templates but will still use the default SDK (which is 9 in this case). Without multi-targeting support in 9, you'd be able to create the template using that ENV but then have to add a global.json, correct? Including @baronfel and @dsplaisted for comment

As to the original issue, this was what we wanted documented per the tactics discussion for now until we have a chance to explore better long-term options. Is there a known issue filed yet covering it?

[joperezr]

This would be awesome, @phenning. I do agree with your previous points that we should also make the 9.0 workload of aspire to also support 8.0 Aspire apps and templates, but I think that we should do a multiple-solution approach that both does this, plus adds your environment variable so that users have different options. I know environment variables are not very discoverable or user friendly, but I think that they are at least a step in the right direction for allowing customers to choose how things get resolved.

[joperezr]

As to the original issue, this was what we wanted documented per the tactics discussion for now until we have a chance to explore better long-term options. Is there a known issue filed yet covering it?

Not yet, I wanted to first see our options with you and @phenning, and sounds like we are likely going to get some workarounds in 17.10 P2 which is great.

[phenning]

If we already had some surface area in Tools/Options for the Template Engine host in VS, I'd say we could add it there instead of env var, but I'm hesitant to add a config page/section for just this.

[phenning]

@marcpopMSFT to your point about global.json, I just tried this.

With NET 9 SDK installed and the ENV set to point to NET 8.
I was able to create a net8.0 API app, add aspire, and build.

It build with the NET 9.0 SDK, but then was runnable and loaded the NET 8 assemblies and runtime, so global.json is optional in this case, I think.

[phenning]

@joperezr The environment variable workaround is in, but I have a better one.

I currently have a change in flight for 17.10 preview 2 which will allow for multiple versions of the same template pack to be installed, either by the SDK locator, or via dotnet new.

With this new change, we can work around the issue by running

dotnet new install [path to 8.0 template pack under PF\dotnet\template_packs]

Keep in mind that templates installed this way will have precedence over those installed from the SDK, so it is critical to run the uninstall command after the manifest updates get into the product.

Or, via manifest update, these are changes which could be made today with the existing names and identities of the template packs: This assumes that there are template packs called Aspire.ProjectTemplates.9.0.0-preview.1.24113.3 and Aspire.ProjectTemplates.8.0.0-preview.3.24105.21 on disk. This requires the change I just made in WebTools due to the template packs being passed to us have the same identity, and we would normally just take the highest versioned template pack. Allowing this was a simple configuration change in the WebTools host. The challenge here is keeping the net8 version number in sync in the net 9 manifest.

"Aspire.ProjectTemplates.net9": {
   "kind": "template",
   "version": "9.0.0-preview.1.24113.3",
   "alias-to": {
     "any": "Aspire.ProjectTemplates"
   }
 },
 "Aspire.ProjectTemplates.net8": {
   "kind": "template",
   "version": "8.0.0-preview.3.24105.21",
   "alias-to": {
     "any": "Aspire.ProjectTemplates"
   }
 },

Alternatively, when/f the template packs are renamed to indicate net8 vs net9, we can drop the 'alias-to', these assume that This assumes that there are template packs called Aspire.ProjectTemplates.net9.9.0.0-preview.1.24113.3 and Aspire.ProjectTemplates.net8.8.0.0-preview.3.24105.21 on disk. This manifest change would not have needed the workaround I just checked into WebTools, as the nupkg identities would have actually been distinct.

    "Aspire.ProjectTemplates.net9": {
      "kind": "template",
      "version": "9.0.0-preview.1.24113.3"
    },
    "Aspire.ProjectTemplates.net8": {
      "kind": "template",
      "version": "8.0.0-preview.3.24105.21"
    },

[balachir]

Applied preview 4 milestone for consideration. Please triage and move it out if needed.

Note: As Phil mentions above, we have a workaround for this issue now.

[joperezr]

@phenning can you share what the environment variable is and what should the value to set it to? (Path to SDK to use to resolve, SDK version to use, something else?)

The dotnet new install trick is also good to have, but as you pointed out, it does feel like a more advanced scenario which can lead to breaks in the future if folks forget to uninstall the packs, so perhaps we should stick with the env variable as the documented workaround?

[phenning]

SET VS_TEMPLATELOCATOR_SDKVERSION=version

Where version is any of:

dotnet --info

.NET SDKs installed:
  6.0.419 [C:\Program Files\dotnet\sdk]
  7.0.406 [C:\Program Files\dotnet\sdk]
  8.0.100-rtm.23523.3 [C:\Program Files\dotnet\sdk]
  8.0.200 [C:\Program Files\dotnet\sdk]
  8.0.300-preview.24101.10 [C:\Program Files\dotnet\sdk]
  9.0.100-preview.1.24101.2 [C:\Program Files\dotnet\sdk]

for example (via cmd shell)
SET VS_TEMPLATELOCATOR_SDKVERSION=8.0.300-preview.24101.10

When this is set, there will be an additional entry in the VS Activity log similar to the following. This should help in diagnosing issues where people get "stuck" on an unexpected version.
sdk version override: 8.0.300-preview.24101.10

[joperezr]

There is nothing really actionable for this issue so moving out of P5 milestone. We'll keep it open in case people hit this issue as a refcount and so that they can find the workaround (env variable) to set.