[Umbrella] Improving compiler adoption and dogfooding story

[jcouv]

See the various sections below for adopting new versions of the compiler (including nightlies) in various contexts (VS, ASP.NET, etc). This is still work-in-progress.

As we shift to point release model, it is increasingly important that we make it easy for customers (external and internal to Microsoft) to adopt new versions of the compiler, including previews.

Host	Users can upgrade?	Upgraded by default?	Allows dogfooding	Plan
VS	Yes, install VSIX. But un-install problem	Yes. Default is “default” (not “latest”). Lighbulb helps upgrade existing projects	Somewhat (but no channel for Roslyn/master yet). Roslyn insider extension currently broken.	Fix SDK un-install issue. Could we get a channel with Roslyn/master?
asp.net	Manually (update compiler nuget and fix web.config).	Yes (new projects in VS2017 use C# 7.0). No for existing projects.		Maybe we could offer lightbulb too? Upgrade the default package reference to 2.0? The template was recently updated to use "default".
asp.net Core	Manually (update compiler nuget, fix web.config and configure Razor). (aspnet/Razor#1046)	Yes (new projects in VS2017 use C# 7.0). No for existing projects.		Maybe we could offer lightbulb too? (filed #20614)
CLI	Yes (see below)	Not sure.	No.	Use package integration. .Net Core 2.0 should offer C# 7.0. Not sure about 7.1. Matt Whilden updated the package recently to 2.3 preview. Jared is tracking an issue to improve the integration.
corext				Moving to package dependency. See steps below (still being validated)
Mono	No. Mono still using own compiler			Mono planning to adopt Roslyn compiler for Mono 5.0. Upgrading from there has package issue.
VS for Mac	Not sure	Yes. Latest preview includes C# 7.0	Not sure	Same package issue as Mono.
Unity	They depend on mono using Roslyn first
Azure functions		Consumes nuget package. Likely uses "default" language version.		Fabio will update to C# 7.0 package. After that we should discuss using "latest".
Powershell	I don't think so
Linqpad	Not sure	Yes (Linqpad seems to pick up new compiler quickly)	No
RoslynPad
OzCode
VS Code	No, new features require some integration work (syntax highlighting). But Omnisharp and VS Code pick up new compiler regularly.			Talk with Dustin
Orleans
VSTS				Jared mentioned there are two scenarios: CI images and some code analysis services

Then there's also internal adoption and dogfooding: building VS code itself, building corefx, building corefxlab, dogfooding with .NET Native, etc

There is also the Microsoft.CodeDom.Providers.DotNetCompilerPlatform package, which has some relationship to ASP.NET.

[jcouv]

Consuming C# compiler from CoreXT

MsBuild 14

1. Edit your \.corext\corext.config
   a. In the <repos> section, ensure you have a <repo> with uri= https://ossmsft.pkgs.visualstudio.com/_packaging/OSS_All/nuget/v3/index.json
   b. In the <packages> section below MsBuild.Corext, add <package id="Microsoft.Net.Compilers" version="1.3.2" />
   c. Below that, add <package id="Microsoft.Net.Compilers.MsBuild14.OnCorext" version="1.0.0" />
2. Edit your \build\msbuild\common\build.props
   a. (if you don't have this file, see https://microsoft.sharepoint.com/teams/corext/LivingDocs/CustomizeYourBranches.aspx )
   b. Add <Import Project="$(PkgMicrosoft_Net_Compilers)\build\Microsoft.Net.Compilers.props" />
3. Close and open your enlistment (or type 'rewind' if you have package Corext.Rewind)
4. build

Note: this doesn't work with Microsoft.Net.Compilers version 2.0 (which supports C# 7.0). The main reason is that the file layout that integrates the compiler into MSBuild has changed. Also, there may also be a framework requirement reason.

MsBuild 15

1. Edit your \.corext\corext.config
   a. In the <repos> section, ensure you have a <repo> with uri= https://ossmsft.pkgs.visualstudio.com/_packaging/OSS_All/nuget/v3/index.json
   b. In the <packages> section add <package id="Microsoft.Net.Compilers" version="2.0.1" />
2. Edit your \build\msbuild\common\build.props
   a. (if you don't have this file, see https://microsoft.sharepoint.com/teams/corext/LivingDocs/CustomizeYourBranches.aspx )
   b. Add <Import Project="$(PkgMicrosoft_Net_Compilers)\build\Microsoft.Net.Compilers.props" />
3. Close and open your enlistment (or type 'rewind' if you have package Corext.Rewind)
4. build

(Thanks Zack Runner for the instructions and package)

[jcouv]

Dogfooding Roslyn on VS 2017

Important note: there is a known issue with un-installing this. Use at your own perils. We are working towards a solution.

Installation instructions

1. Start VS 2017
2. Go to “Tools\Options\General\Extensions and Updates”
3. Click “Add”, and enter “Roslyn Nightly on MyGet” for the name and https://dotnet.myget.org/F/roslyn/vsix for the URL:

4. If you want the extension to auto-update when you start VS, make sure the checkboxes at the top are checked.
5. Click “Apply”, and then “OK”
6. Go to Tools\Extensions and Updates\
7. Click “Online” and then “Roslyn Nightly on MyGet” in the left hand side.
8. Choose “Roslyn Insiders for VS next” and click “Download” and “Close”

9. Close all instances of VS
10. Click “Modify” on the VSIX Installer window that starts:

11. Wait for installation to complete and click “Close”
12. Start VS
13. You’re now using the latest Roslyn binaries!

To get back to a clean state:

1. Go to %LocalAppData%\Microsoft\VisualStudio\15.0_\ and delete:
   a. The “Extensions” directory
   b. The “ComponentModelCache” directory
   c. “devenv.exe.config”
2. Start a Developer command prompt and run “devenv /updateconfiguration”
3. Start VS.
4. You should be back to normal.

[jcouv]

Relates to #13361 (letting users update the compiler in CLI)

[jcouv]

My list of contacts:

• ASP.Net: Matt Fei, Damian Edwards (and Taylor Mullen, Eilon Lipton), Ryan Nowak (Razor)
• CLI: Livar Cunha
• coreXT: Zack Runner, Newton Sanches
• CloudBuild/QuickBuild: Newton Sanches, David Federman, Hamed Esfahani
• Domino: Danny van Velzen
• Mono, VS for Mac: Marek Safar
• Azure functions: Fabio Cavalcante, Paul Batum
• VS Code: Dustin Campbell
• Windows: Kevin Burrows
• Orleans: Reuben Bond, Sergey Bykov

[jcouv]

Consuming C# compiler from ASP.NET

A number of people on stackoverflow are reporting trouble adopting C# 7.0 in their ASP.NET and ASP.NET Core projects. This often has strange symptoms, where the IDE accepts the new syntax (no squiggles), but building the solution or F5 produce errors in the Output window.

This is caused by a mismatch between the built-in compiler (used by VS) and the compiler pulled in as a nuget package by ASP.NET projects. The Microsoft.CodeDom.Providers.DotNetCompilerPlatform package pulls in the Microsoft.Net.Compilers package, which overwrites what compiler will be used to build the solution.

To troubleshoot this, open the "Manage NuGet packages" UI, then select the "Browse" tab. Search for "Microsoft.Net.Compilers" and look at the installed version and the newest available version.
Versions 1.x of the package correspond to C# 6.0. You need version 2.x for C# 7.0.

The fix for ASP.NET:

1. Install the latest Microsoft.Net.Compilers package (see versioning information about compiler packages to help you figure out which one you need)
2. Look for "LangVersion" in your web.config and change it to the version you want (for example 7.0)

The fix for ASP.NET Core 1.0:

1. Install the Microsoft.CodeAnalysis.CSharp (version 2.0.0) packages
2. In Startup.cs, in the ConfigureServices method, configure Razor to use C# 7.0 by doing the following:

services.AddMvc().AddRazorOptions(options =>
     options.ParseOptions = new CSharpParseOptions(LanguageVersion.CSharp7));

For ASP.NET Core 2.0, there should be no need to set Razor options. It uses the language version from the project (the Razor views and the code-behind use the same language version). This works with "default" and "latest", but there is a parsing bug with handling "7.1" (tracking issue).

Note: if you want to use C# 7.0 tuples in your project, you'll also need to install System.ValueTuple (version 4.3.0).

Where to find langVersion in web.config? It should look like this:

        ...
        <compilers>
          <compiler language="c#;cs;csharp" extension=".cs" type="Microsoft.CSharp.CSharpCodeProvider, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" warningLevel="4" compilerOptions="/langversion:5/nowarn:1659;1699;1701">
            <providerOption name="CompilerVersion" value="v4.0"/>
          </compiler>
          <compiler language="vb;vbs;visualbasic;vbscript" extension=".vb" type="Microsoft.VisualBasic.VBCodeProvider, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" warningLevel="4" compilerOptions="/langversion:14 /nowarn:41008 /define:_MYTYPE=\&quot;Web\&quot; /optionInfer+">
            <providerOption name="CompilerVersion" value="v4.0"/>
          </compiler>
        </compilers>
        ...

[svick]

Consuming C# compiler from CLI

As far as I can tell, upgrading the dotnet CLI to a nightly version of Roslyn for a single project is not hard (tested with .Net Core SDK 1.0.4 on Windows 10):

1. Add a NuGet.config file referencing the Roslyn MyGet:

   <?xml version="1.0" encoding="utf-8"?>
   <configuration>
     <packageSources>
       <add key="roslyn" value="https://dotnet.myget.org/F/roslyn/api/v3/index.json" />
     </packageSources>
   </configuration>

2. Add a reference to a nightly version of Microsoft.Net.Compilers, e.g.: dotnet add package Microsoft.Net.Compilers -v 3.0.0-dev-61717-03.

3. Add <LangVersion>Latest</LangVersion> to the csproj.

4. Use C# 7.1 features.

[jcouv]

Consuming local C# compiler in MSBuild

If you have a local build of the C# compiler, you can pass its path to msbuild with /p:CscToolExe=...path-to-exe....
This is useful for troubleshooting bootstrapping issues.

[PathogenDavid]

@jcouv Sorry to ping such an old thread, is there any updated guidance on using daily builds with Visual Studio? I use Microsoft.Net.Compilers.Toolset regularly to get the bleeding edge, but that doesn't help the IDE side of things.

It looks like Roslyn Insiders for VS Next hasn't been updated since 2017 unless there's a new location that I'm not aware of. (The VSIX stuff is still on MyGet, right? Only NuGet moved to Azure DevOps?)

Back in June I had a somewhat workable solution by manually installing the Roslyn Language Services, Roslyn Expression Evaluators, and Roslyn Compilers packages. But I remember getting them to install was troublesome and it seemed like I was shoving a square peg in a round hole. It also caused my 2019 Preview install to violently combust upon updating to the next preview came out.

[jcouv]

@PathogenDavid Unfortunately, the best I would recommend is just staying on recent previews. They are not as fresh as daily, but they are less than a month old... :-/ Alternatively, you can set up from a local build.
Tagging @jmarolf who knows the setup aspects better than me.

[PathogenDavid]

That's too bad, but thanks for confirming!

I actually briefly looked into using a local build with -deployExtensions, but I ran into the same issue you did a few days ago. (Plus I was wanting to consume things from CI if possible for the sake of lower friction, which led to me finding this issue.)

[jmarolf]

@PathogenDavid There is no consistently tested mechanism to install the IDE features. Basically you are installing Visual Studio N + 1 bit on Visual Studio N.

You can look at the artifacts that are produced by CI

That zip file will contain a script that attempts to install the latest VSIX into our chosen VS (you will need to launch with devenv /rootSuffix Exp) but it can horribly fail. The most likely scenario to work is to install into the latest preview.

[PathogenDavid]

Thanks for the info! I really appreciate it.

Horribly fail does seem to be accurate:

Well...maybe not horribly. Despite the angry golden bars, it actually worked at a in my quick test. (No error squiggles, but code completion works!) I mostly wanted this to confirm IDE bugs weren't already fixed before I report them (and if it worked well, there's been fixes I'd definitely appreciate having with a preview 3+n.) This might be good enough for the former, but not the latter.

[jmarolf]

Those gold bars are about the OOP server not being updated but we fall back to doing everything in-proc for this case so things should still just work™

[PathogenDavid]

Ah, I was kinda surprised it still mostly worked after an error like that but that makes sense. Thanks for the insight!