Question: getting Project instances in a .NET core workspace

[kentcb]

Hi,

I'm porting code that uses types in Microsoft.CodeAnalysis to the new .NET core world. I've already hit up against #17439, and have pulled the relevant code from Omnisharp. However, I now need to figure out how to bridge the gap between the old world and new.

From what I can tell, there is no compatibility between Microsoft.DotNet.ProjectModel and Microsoft.CodeAnalysis. I'm hoping I'm wrong about that.

As an example, how do I get a list of Microsoft.CodeAnalysis.Project objects for in a DotNetWorkspace? I suspect if I can bridge that gap, I'll be good.

[Pilchie]

Also tagging @DustinCampbell - I haven't looked at DotNetWorkspace :(, but if you have something that derives from Workspace, then CurrentSolution.Projects should get you the projects...

[kentcb]

Thanks @Pilchie. The problem is that DotNetWorkspace inherits from Microsoft.DotNet.ProjectModel.Workspace, which does not have a CurrentSolution property.

I'd be very grateful for your thoughts, @DustinCampbell.

[DustinCampbell]

@kentcb: The problem is that Microsoft.DotNet.ProjectModel is not actually a Roslyn artifact, so the Roslyn team doesn't know anyway thing about them. There isn't really a relationship between them. There is Microsoft.DotNet.ProjectModel.Workspaces which contains a ProjectJsonWorkspace that you might find interesting or helpful.

In OmniSharp, DotNetWorkspace is used to populate a proper Roslyn workspace in OmniSharp's DotNetProjectSystem.

Note that project.json is officially deprecated with the release of VS 2017. Microsoft.DotNet.ProjectModel and Microsoft.DotNet.ProjectModel.Workspaces were last updated in November and may not continue to work in the future as newer versions of their dependencies (like NuGet and Roslyn).

[kentcb]

Thanks @DustinCampbell

This, then, is my exact question. Given that I'm trying to port my code to the new world, I'm kinda stuck without official support. Do you have timelines for when this will become available?

[kentcb]

Oh, I just saw that @Pilchie assigned to 15.3 milestone. Given it's in its infancy, I'm guessing it's unrealistic to ask for dates at this point. I will keep an eye on it as it progresses. Thanks again.

[DustinCampbell]

I replied on Twitter as well. This is actually a pretty big work item with a bunch of unknowns because the functionality depends on what runtime MSBuildWorkspace will be running on. For example, if it is running on CoreCLR in a .NET Core project, MSBuildWorkspace won't be able to process many .NET Framework projects (due to those projects' MSBuild Tasks needing to run on .NET Framework). It's a very sticky problem.

[kentcb]

@DustinCampbell thanks for the further clarification. And, yes, that does sound nasty!

This is an admission of my own ignorance, but I thought that since VS2017 had shipped with the new project model, all the work was done. My very high-level understanding was that VS2017 uses Roslyn behind the scenes for all compilation and solution/project management. Therefore, I figured the workspace already existed somewhere. It's now apparent to me that this is not the case, though I still don't understand why.

[DustinCampbell]

VS 2017 has a new project system (for .NET Core projects only) that is thoroughly tied to Visual Studio. This is not at the same/level abstraction as a Roslyn workspace and is much more complex than what is needed for MSBuildWorkspace.

MSBuildWorkspace is an API that allows you to load an MSBuild solution/project(s) into a Roslyn workspace. The scenario here is someone who wants to perform analysis on an existing solution or project. It doesn't offer several features that a project system would, like mutation. For example, within an MSBuildWorkspace there isn't a way to add a new project because MSBuildWorkspace doesn't know how to generate or manipulate MSBuild files. Does that make sense?

If you want something more advanced, you can use the MSBuild APIs and populate a custom workspace, much like OmniSharp does.

[kentcb]

Makes sense, @DustinCampbell

perform analysis on an existing solution or project

This is exactly what I need it for. PCLMock (the project I'm porting) has a code generator that wants to load the user's solution, examine it, and generate mocks according to what it finds - no need for mutation. This works 💯 with MSBuildWorkspace in the old world, so am wanting to swap in a workspace that works in the new world and have all the existing code just work.

[josephwoodward]

If you want something more advanced, you can use the MSBuild APIs and populate a custom workspace, much like OmniSharp does.

Out of interest @DustinCampbell, how difficult/easy is populating a custom workspace? I presume it would involve including binaries from sources such as local NuGet repositories etc, which seems like a lot of work?

[jskeet]

Like @kentcb, I'm in the world of "I already have a project, I just want to load and analyze it."
(In fact I'd like to mess with the syntax trees of some source files and remove some statements, then save to a new filename, but that can come later.)

I'm able to load a project into an MSBuildWorkspace (workspace.OpenProjectAsync) but the result never has any documents, and the compilation retrieved with GetCompilationAsync() doesn't have any syntax trees.

@DustinCampbell it wasn't quite clear to me from your previous comment whether you'd expect all that to work today using Microsoft.Build v15.1.1012 and Microsoft.CodeAnalysis.* v2.1.0 (on a regular Windows machine with all full-fat frameworks installed) or whether that's the scenario you're expecting to come to fruition over time. (I'm quite happy to wait - I have no shortage of other things I really should be doing...)

[DustinCampbell]

@jskeet: I think it's possible, but I haven't investigated what's required to make it work. There are a number of challenges that MSBuild 15 and VS 2017 bring to the table:

1. Using MSBuildWorkspace in a .NET Core app may only work for a particular set of projects because the targets, tasks, Sdks, etc. required by a particular project may not work on CoreCLR.
2. Getting MSBuildWorkspace to locate the correct set of targets, tasks, Sdks, etc. expected by a project is tricky because there may be multiple versions of VS 2017 installed side-by-side, and they might not all have the same workloads installed.
3. Making it all work cross-platform is even messier.

This is essentially the problem that I've been working to solve with OmniSharp, and it isn't pretty.

[jskeet]

@DustinCampbell: For the minute, I'm going to experiment with an old-school MSBuild project - that way I can at least attack my Roslyn-ignorance. Once I understand a bit more about how it's meant to fit together, I can try on a .NET Core project again...

[DustinCampbell]

Note that there are two pivots here: what sort of application you are using MSBuildWorkspace within (.NET Core or .NET Framework) and what sort project you're trying to analyze (.NET Core or .NET Framework).

[jskeet]

Ack. I don't mind being restricted to doing it from a full Framework application, but I'd like to analyze a Core project. No idea what others want though :)

[DustinCampbell]

Using MSBuildWorkspace from a full Framework application does not restrict you from analyzing .NET Core projects. However, using MSBuildWorkspace from a .NET Core application may restrict you from analyzing full Framework projects.

[jskeet]

I can't get MSBuildWorkspace to work at all in a .NET Core application at the moment - but this snippet is working in a net46 console app loading an "old school" class library csproj, but not loading a .NET Core csproj:

var workspace = MSBuildWorkspace.Create();
var project = await workspace.OpenProjectAsync(projectFile);

foreach (var doc in project.Documents)
{
    Console.WriteLine(doc.FilePath);
}

It doesn't throw any exceptions when loading the project - it just doesn't have any documents, and if I call GetCompilationAsync the compilation doesn't have any syntax trees.

I see the same behaviour when I change the application to a "classic" desktop console app targeting .NET 4.6.

Given the MEF aspects to it, might this be due to some packages not being installed? This is all I need to get the analysis of the regular .NET class library working:

<ItemGroup>
  <PackageReference Include="Microsoft.Build" Version="15.1.1012" />
  <PackageReference Include="Microsoft.Build.Tasks.Core" Version="15.1.1012" />
  <PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="2.1.0" />
  <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="2.1.0" />
  <PackageReference Include="Microsoft.CodeAnalysis.Workspaces.Common" Version="2.1.0" />
</ItemGroup>

[DustinCampbell]

It sounds to me like tasks/targets aren't being run properly. I believe @mattwar added a way to get MSBuild failures when opening a project or solution. Maybe check the MSBuildWorkspace.Diagnostics property?

[jskeet]

Nothing in workspace.Diagnostics, no... and it may be relevant that it has worked out the other project that the library I'm analyzing requires. (I'm analyzing NodaTime.Demo.csproj, which has a project reference to NodaTime.csproj; that dependency is present in the loaded project.)

I didn't mean to derail this issue though - if it's likely to be helpful to anyone else, I'm definitely happy to keep trying things (including setting up a github branch somewhere with all of this code easily available) but I don't want to suck time away from more important things if it's only going to help me :)

[Pilchie]

Note that you'll need to use the 2.3-* packages from our MyGet feed for the Diagnostics events to be hooked up. It happened after our most recently released packages were frozen.

[josephwoodward]

I echo @kentcb's sentiment 👎

Would be interested in knowing how easy/hard it is to create a custom workspace (similar to OmniSharps) that could be open-sourced in the meantime.

[kentcb]

@jskeet Would you mind verifying my understanding of your situation? I am wondering whether I need to take a two-phased approach to solving this in PCLMock, given that this issue has been pushed out to "who knows when", namely:

1. Update my existing FX-based code generators to support loading netstandard projects when generating code
2. Update PCLMock itself to netstandard once this issue is fixed

Here's my understanding of your situation...you are running under desktop FX using latest pre-release Roslyn packages. In addition, you had to add a dependency on Microsoft.CodeAnalysis.Build.Tasks (which I actually can't find in NuGet - perhaps it's been delisted?). If a project multitargets, I'll need to specify a moniker when opening the project.

Can you confirm? Anything I'm missing here? What is Microsoft.CodeAnalysis.Build.Tasks?

Thanks

[jskeet]

@kentcb: In fact I haven't managed to get it to load the netstandard version yet - I just target the net45 build at the moment.

I'm using prereleases on MyGet - that may be why you can't find Microsoft.CodeAnalysis.Build.Tasks. My code and project file are all available here:

https://github.com/nodatime/nodatime/tree/master/build/SnippetExtractor

I've got the myget feed in my NuGet sources.

[josephwoodward]

@Pilchie Do you know if there's been any progress on support for this yet?

[Pilchie]

@josephwoodward Is this issue still open?

[alexsorokoletov]

@Pilchie
I'm not a topic started, however, follow this issue long time. Not sure if that's the goal of the issue, but I'm interested in having a sane way to load .NET Core .csproj file with all documents and stuff. Pretty much MSBuildWorkspace available for .NET Core.

So this issue for me is basically the main place to follow any progress on this.
Suggestions from @DustinCampbell are extremely helpful but it is still hard to do and with MSBuildWorkspace pushed to later milestone, maybe we could have some alternative?

[jskeet]

I've been looking at removing the MyGet dependency from the Noda Time build - I'd hoped that now that Roslyn 2.3 is out, I could use the nuget.org packages. Unfortunately, Microsoft.CodeAnalysis.Build.Tasks is still only on MyGet as far as I can see. Without that, I'm back to framework types not being found.

Is there an expectation that this should work now? Very to happy to try to repro in a minimal example if that would be useful.

[DustinCampbell]

Microsoft.CodeAnalysis.Build.Tasks is included in Microsoft.Net.Compilers

[jskeet]

Humbug - that wasn't enough to get it working. Will try to come up with a minimal example - ideally one that works with the beta packages that Noda Time still uses, but doesn't work with the released ones - and then we can play spot the difference.

[josephwoodward]

For anyone coming across this looking for cross-platform support for theMSBuildWorkspace then whilst it's still not here yet and keeps getting moved back, there is a library recently released called Buildalyzer that can create an AdhocWorkspace allowing you to hopefully achieve the same goal.

[giggio]

Buildalyzer suggested by @josephwoodward works! Just a heads up to anyone reading, if your analyzed project (or analyzer project, I am not sure) is on .NET Core Sdk 2.1 preview 1 it won't work. Just go back to 2.0 and it works.
I would love to see Microsoft finally finish this. It's been quite a while. It's sad to see it pushed to the Unknown milestone.

[daveaglick]

@giggio A little more info on that: Buildalyzer gets updated when new final SDKs are released since new MSBuild packages are usually published at the same time. Unfortunately, when an SDK preview gets out of sync with the previously released MSBuild packages, Buildalyzer can have a hard time since it will locate the preview SDK bits but be using the previously released MSBuild bits from the package.

TL;DR: Buildalyzer may be unstable on systems where the latest SDK is a preview, but will (hopefully) work on systems where the latest SDK is a final release (with a couple days lag for me to get a new version out for each SDK).

[karlingen]

Ping

[queil]

Hey, is there any progress on this issue?

[Meowzz95]

Hey.. last comment was a year ago.. Should we expect to get this soon?

[CyrusNajmabadi]

@Meowzz95 I don't believe there's be any progress on this

[Meowzz95]

@CyrusNajmabadi
Sad to hear that, hopefully we can count on .net 5.

[CyrusNajmabadi]

Sad to hear that, hopefully we can count on .net 5.

I think we'd take a community contribution here if you're interested :)

[Meowzz95]

Sad to hear that, hopefully we can count on .net 5.

I think we'd take a community contribution here if you're interested :)

I would like to do so but I'm afraid my skill isn't up for this...yet... :)