Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Umbrella] Improving compiler adoption and dogfooding story #18783

Open
jcouv opened this issue Apr 18, 2017 · 14 comments
Open

[Umbrella] Improving compiler adoption and dogfooding story #18783

jcouv opened this issue Apr 18, 2017 · 14 comments

Comments

@jcouv
Copy link
Member

@jcouv jcouv commented Apr 18, 2017

See the various sections below for adopting new versions of the compiler (including nightlies) in various contexts (VS, ASP.NET, etc). This is still work-in-progress.

As we shift to point release model, it is increasingly important that we make it easy for customers (external and internal to Microsoft) to adopt new versions of the compiler, including previews.

Host Users can upgrade? Upgraded by default? Allows dogfooding Plan
VS Yes, install VSIX. But un-install problem Yes. Default is “default” (not “latest”). Lighbulb helps upgrade existing projects Somewhat (but no channel for Roslyn/master yet). Roslyn insider extension currently broken. Fix SDK un-install issue. Could we get a channel with Roslyn/master?
asp.net Manually (update compiler nuget and fix web.config). Yes (new projects in VS2017 use C# 7.0). No for existing projects. Maybe we could offer lightbulb too? Upgrade the default package reference to 2.0? The template was recently updated to use "default".
asp.net Core Manually (update compiler nuget, fix web.config and configure Razor). (aspnet/Razor#1046) Yes (new projects in VS2017 use C# 7.0). No for existing projects. Maybe we could offer lightbulb too? (filed #20614)
CLI Yes (see below) Not sure. No. Use package integration. .Net Core 2.0 should offer C# 7.0. Not sure about 7.1. Matt Whilden updated the package recently to 2.3 preview. Jared is tracking an issue to improve the integration.
corext Moving to package dependency. See steps below (still being validated)
Mono No. Mono still using own compiler Mono planning to adopt Roslyn compiler for Mono 5.0. Upgrading from there has package issue.
VS for Mac Not sure Yes. Latest preview includes C# 7.0 Not sure Same package issue as Mono.
Unity They depend on mono using Roslyn first
Azure functions Consumes nuget package. Likely uses "default" language version. Fabio will update to C# 7.0 package. After that we should discuss using "latest".
Powershell I don't think so
Linqpad Not sure Yes (Linqpad seems to pick up new compiler quickly) No
RoslynPad
OzCode
VS Code No, new features require some integration work (syntax highlighting). But Omnisharp and VS Code pick up new compiler regularly. Talk with Dustin
Orleans
VSTS Jared mentioned there are two scenarios: CI images and some code analysis services

Then there's also internal adoption and dogfooding: building VS code itself, building corefx, building corefxlab, dogfooding with .NET Native, etc

There is also the Microsoft.CodeDom.Providers.DotNetCompilerPlatform package, which has some relationship to ASP.NET.

@jcouv jcouv changed the title Improving compiler adoption and dogfooding story [Umbrella] Improving compiler adoption and dogfooding story Apr 18, 2017
@jcouv
Copy link
Member Author

@jcouv jcouv commented Apr 21, 2017

Consuming C# compiler from CoreXT

MsBuild 14

  1. Edit your \.corext\corext.config
    a. In the <repos> section, ensure you have a <repo> with uri= https://ossmsft.pkgs.visualstudio.com/_packaging/OSS_All/nuget/v3/index.json
    b. In the <packages> section below MsBuild.Corext, add <package id="Microsoft.Net.Compilers" version="1.3.2" />
    c. Below that, add <package id="Microsoft.Net.Compilers.MsBuild14.OnCorext" version="1.0.0" />
  2. Edit your \build\msbuild\common\build.props
    a. (if you don't have this file, see https://microsoft.sharepoint.com/teams/corext/LivingDocs/CustomizeYourBranches.aspx )
    b. Add <Import Project="$(PkgMicrosoft_Net_Compilers)\build\Microsoft.Net.Compilers.props" />
  3. Close and open your enlistment (or type 'rewind' if you have package Corext.Rewind)
  4. build

Note: this doesn't work with Microsoft.Net.Compilers version 2.0 (which supports C# 7.0). The main reason is that the file layout that integrates the compiler into MSBuild has changed. Also, there may also be a framework requirement reason.

MsBuild 15

  1. Edit your \.corext\corext.config
    a. In the <repos> section, ensure you have a <repo> with uri= https://ossmsft.pkgs.visualstudio.com/_packaging/OSS_All/nuget/v3/index.json
    b. In the <packages> section add <package id="Microsoft.Net.Compilers" version="2.0.1" />
  2. Edit your \build\msbuild\common\build.props
    a. (if you don't have this file, see https://microsoft.sharepoint.com/teams/corext/LivingDocs/CustomizeYourBranches.aspx )
    b. Add <Import Project="$(PkgMicrosoft_Net_Compilers)\build\Microsoft.Net.Compilers.props" />
  3. Close and open your enlistment (or type 'rewind' if you have package Corext.Rewind)
  4. build

(Thanks Zack Runner for the instructions and package)

@jcouv
Copy link
Member Author

@jcouv jcouv commented May 3, 2017

Dogfooding Roslyn on VS 2017

Important note: there is a known issue with un-installing this. Use at your own perils. We are working towards a solution.

Installation instructions

  1. Start VS 2017
  2. Go to “Tools\Options\General\Extensions and Updates”
  3. Click “Add”, and enter “Roslyn Nightly on MyGet” for the name and https://dotnet.myget.org/F/roslyn/vsix for the URL:

1

  1. If you want the extension to auto-update when you start VS, make sure the checkboxes at the top are checked.
  2. Click “Apply”, and then “OK”
  3. Go to Tools\Extensions and Updates\
  4. Click “Online” and then “Roslyn Nightly on MyGet” in the left hand side.
  5. Choose “Roslyn Insiders for VS next” and click “Download” and “Close”

2

  1. Close all instances of VS
  2. Click “Modify” on the VSIX Installer window that starts:

3

  1. Wait for installation to complete and click “Close”
  2. Start VS
  3. You’re now using the latest Roslyn binaries!

To get back to a clean state:

  1. Go to %LocalAppData%\Microsoft\VisualStudio\15.0_\ and delete:
    a. The “Extensions” directory
    b. The “ComponentModelCache” directory
    c. “devenv.exe.config”
  2. Start a Developer command prompt and run “devenv /updateconfiguration”
  3. Start VS.
  4. You should be back to normal.
@jcouv
Copy link
Member Author

@jcouv jcouv commented May 28, 2017

Relates to #13361 (letting users update the compiler in CLI)

@jcouv
Copy link
Member Author

@jcouv jcouv commented May 30, 2017

My list of contacts:

  • ASP.Net: Matt Fei, Damian Edwards (and Taylor Mullen, Eilon Lipton), Ryan Nowak (Razor)
  • CLI: Livar Cunha
  • coreXT: Zack Runner, Newton Sanches
  • CloudBuild/QuickBuild: Newton Sanches, David Federman, Hamed Esfahani
  • Domino: Danny van Velzen
  • Mono, VS for Mac: Marek Safar
  • Azure functions: Fabio Cavalcante, Paul Batum
  • VS Code: Dustin Campbell
  • Windows: Kevin Burrows
  • Orleans: Reuben Bond, Sergey Bykov
@jcouv jcouv self-assigned this Jun 7, 2017
@jcouv
Copy link
Member Author

@jcouv jcouv commented Jun 14, 2017

Consuming C# compiler from ASP.NET

A number of people on stackoverflow are reporting trouble adopting C# 7.0 in their ASP.NET and ASP.NET Core projects. This often has strange symptoms, where the IDE accepts the new syntax (no squiggles), but building the solution or F5 produce errors in the Output window.

This is caused by a mismatch between the built-in compiler (used by VS) and the compiler pulled in as a nuget package by ASP.NET projects. The Microsoft.CodeDom.Providers.DotNetCompilerPlatform package pulls in the Microsoft.Net.Compilers package, which overwrites what compiler will be used to build the solution.

To troubleshoot this, open the "Manage NuGet packages" UI, then select the "Browse" tab. Search for "Microsoft.Net.Compilers" and look at the installed version and the newest available version.
Versions 1.x of the package correspond to C# 6.0. You need version 2.x for C# 7.0.

img

The fix for ASP.NET:

  1. Install the latest Microsoft.Net.Compilers package (see versioning information about compiler packages to help you figure out which one you need)
  2. Look for "LangVersion" in your web.config and change it to the version you want (for example 7.0)

The fix for ASP.NET Core 1.0:

  1. Install the Microsoft.CodeAnalysis.CSharp (version 2.0.0) packages
  2. In Startup.cs, in the ConfigureServices method, configure Razor to use C# 7.0 by doing the following:
services.AddMvc().AddRazorOptions(options =>
     options.ParseOptions = new CSharpParseOptions(LanguageVersion.CSharp7));

For ASP.NET Core 2.0, there should be no need to set Razor options. It uses the language version from the project (the Razor views and the code-behind use the same language version). This works with "default" and "latest", but there is a parsing bug with handling "7.1" (tracking issue).

Note: if you want to use C# 7.0 tuples in your project, you'll also need to install System.ValueTuple (version 4.3.0).

Where to find langVersion in web.config? It should look like this:

        ...
        <compilers>
          <compiler language="c#;cs;csharp" extension=".cs" type="Microsoft.CSharp.CSharpCodeProvider, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" warningLevel="4" compilerOptions="/langversion:5/nowarn:1659;1699;1701">
            <providerOption name="CompilerVersion" value="v4.0"/>
          </compiler>
          <compiler language="vb;vbs;visualbasic;vbscript" extension=".vb" type="Microsoft.VisualBasic.VBCodeProvider, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" warningLevel="4" compilerOptions="/langversion:14 /nowarn:41008 /define:_MYTYPE=\&quot;Web\&quot; /optionInfer+">
            <providerOption name="CompilerVersion" value="v4.0"/>
          </compiler>
        </compilers>
        ...
@svick
Copy link
Contributor

@svick svick commented Jun 14, 2017

Consuming C# compiler from CLI

As far as I can tell, upgrading the dotnet CLI to a nightly version of Roslyn for a single project is not hard (tested with .Net Core SDK 1.0.4 on Windows 10):

  1. Add a NuGet.config file referencing the Roslyn MyGet:

    <?xml version="1.0" encoding="utf-8"?>
    <configuration>
      <packageSources>
        <add key="roslyn" value="https://dotnet.myget.org/F/roslyn/api/v3/index.json" />
      </packageSources>
    </configuration>
  2. Add a reference to a nightly version of Microsoft.Net.Compilers, e.g.: dotnet add package Microsoft.Net.Compilers -v 3.0.0-dev-61717-03.

  3. Add <LangVersion>Latest</LangVersion> to the csproj.

  4. Use C# 7.1 features.

imfrancisd added a commit to imfrancisd/yabs that referenced this issue Jun 25, 2017
Remove Microsoft.Net.Compilers.netcore package from build.

  Microsoft.Net.Compilers.netcore needs to be updated for netstandard
  dotnet/roslyn#12674

    agocke commented on Oct 3 2016
    @onovotny Do you think you could give a little background on your
    use case for this? This package was a bit of a hack to just get
    things in the right place for .NET CLI so I'd be curious how other
    people are using it.

  HOWTO use C# 7 with .NET Core
  dotnet/roslyn#13361

    svick commented on Aug 27 2016  edited
    I've managed to figure out a way, but it's super convoluted
    (possibly unnecessarily).

  [Umbrella] Improving compiler adoption and dogfooding story
  dotnet/roslyn#18783

    See the various sections below for adopting new versions of the
    compiler (including nightlies) in various contexts (VS, ASP.NET,
    etc). This is still work-in-progress.

    As we shift to point release model, it is increasingly important
    that we make it easy for customers (external and internal to
    Microsoft) to adopt new versions of the compiler, including
    previews.

    Host        Users can upgrade?  Upgraded by default?  Allows dogfooding  Plan
    .
    .
    .
    Powershell  I don't think so
    .
    .
    .

  Hey! C'mon now. PowerShell user trying to eat your dogfood here.

      One who is full spurns honey;
      but to the hungry, any bitter thing is sweet.
      - Prv 27:7

  Sigh. PowerShell life.
@jcouv jcouv moved this from Active to Wanna fix in Compiler: Julien's umbrellas Sep 22, 2017
@jcouv jcouv added this to the 16.0 milestone Nov 5, 2017
@jcouv
Copy link
Member Author

@jcouv jcouv commented Jan 2, 2019

Consuming local C# compiler in MSBuild

If you have a local build of the C# compiler, you can pass its path to msbuild with /p:CscToolExe=...path-to-exe....
This is useful for troubleshooting bootstrapping issues.

@jcouv jcouv modified the milestones: 16.0, Backlog Jan 6, 2019
@PathogenDavid
Copy link
Contributor

@PathogenDavid PathogenDavid commented Sep 15, 2020

@jcouv Sorry to ping such an old thread, is there any updated guidance on using daily builds with Visual Studio? I use Microsoft.Net.Compilers.Toolset regularly to get the bleeding edge, but that doesn't help the IDE side of things.

It looks like Roslyn Insiders for VS Next hasn't been updated since 2017 unless there's a new location that I'm not aware of. (The VSIX stuff is still on MyGet, right? Only NuGet moved to Azure DevOps?)

Back in June I had a somewhat workable solution by manually installing the Roslyn Language Services, Roslyn Expression Evaluators, and Roslyn Compilers packages. But I remember getting them to install was troublesome and it seemed like I was shoving a square peg in a round hole. It also caused my 2019 Preview install to violently combust upon updating to the next preview came out.

@jcouv
Copy link
Member Author

@jcouv jcouv commented Sep 15, 2020

@PathogenDavid Unfortunately, the best I would recommend is just staying on recent previews. They are not as fresh as daily, but they are less than a month old... :-/ Alternatively, you can set up from a local build.
Tagging @jmarolf who knows the setup aspects better than me.

@PathogenDavid
Copy link
Contributor

@PathogenDavid PathogenDavid commented Sep 15, 2020

That's too bad, but thanks for confirming!

I actually briefly looked into using a local build with -deployExtensions, but I ran into the same issue you did a few days ago. (Plus I was wanting to consume things from CI if possible for the sake of lower friction, which led to me finding this issue.)

@jmarolf
Copy link
Member

@jmarolf jmarolf commented Sep 15, 2020

@PathogenDavid There is no consistently tested mechanism to install the IDE features. Basically you are installing Visual Studio N + 1 bit on Visual Studio N.

You can look at the artifacts that are produced by CI

image

That zip file will contain a script that attempts to install the latest VSIX into our chosen VS (you will need to launch with devenv /rootSuffix Exp) but it can horribly fail. The most likely scenario to work is to install into the latest preview.

@PathogenDavid
Copy link
Contributor

@PathogenDavid PathogenDavid commented Sep 16, 2020

Thanks for the info! I really appreciate it.

Horribly fail does seem to be accurate:

image

Well...maybe not horribly. Despite the angry golden bars, it actually worked at a in my quick test. (No error squiggles, but code completion works!) I mostly wanted this to confirm IDE bugs weren't already fixed before I report them (and if it worked well, there's been fixes I'd definitely appreciate having with a preview 3+n.) This might be good enough for the former, but not the latter.

@jmarolf
Copy link
Member

@jmarolf jmarolf commented Sep 16, 2020

Those gold bars are about the OOP server not being updated but we fall back to doing everything in-proc for this case so things should still just work™

@PathogenDavid
Copy link
Contributor

@PathogenDavid PathogenDavid commented Sep 17, 2020

Ah, I was kinda surprised it still mostly worked after an error like that but that makes sense. Thanks for the insight!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.