f# project system vNext

[enricosada]

I'd like to discuss how can we improve the f# project system next version, when development begins after vs 2015 ship.

This issue is not about asking for a roadmap, but discuss how we can do that.

My list of “Scratch your own itch” about the vs f# project system:

• folders
• show all files
• add class/module

I dont want to speak about azure/universal app/asp.net vNext/etc.

The bugs in my checklist are already fixed in every others vs extension (python, nodejs, rust).

nodejs tools for vs forked the python tools for vs project system, the same MPF codebase used by visualfsharp, but a later version with lot of features and bugfix. It also improved alot

I tried before to import some code from PTVS (the open explorer command) trying to clean and align the source code (see my others open pr) to make it easier to import features and bugfix.

These project are improving really fast, both projects are now on github and use exactly the same source code for shared project system (before these had some differences, like namespace/indentation/some classes ).

The sharedproject was created, to host the shared codebase of the common project system, without specialization for python or node ( a clean namespace Microsoft.VisualStudioTools.Project for example ) and is imported in PTVS and nodejs as git subtree ( see @zooba comment ).

That's a possible way (1) :

• include the sharedproject as submodule
• fix indentation of the current code to make diff easier
• use the new classes (linked item) in submodule code

Another ways (2) is the new Visual Studio Common Project System (CPS)

Lots more info in the repo Visual Studio Project System Extensibility

[enricosada]

@latkin ping about this one.

I'd like to work on that macro issue if possible, can we discuss about it? not only me and you, but a chat session will be nice to bootstrap the discussion maybe

[enricosada]

ok, sharedproj is going in as subtree.

My proposed roadmap to use sharedproj

1. add the source code from shared prj in a directory SHRPRJ ( ref [WIP] Use Microsoft/VisualStudio-SharedProject as subtree #733 )
2. for each old file in VSEXT ( can be done by multiple person/pr, in parallel, easy to review)
   • reformat code style to minimize diff with SHRPRJ (only style changes)
   • if file are equal (binary), link SHRPRJ file in vsproj and remove old code.
3. multiple pr to minimize diff VSEXT and SHRPRJ, really small pr
4. contribute back to sharedproj the visualfsharp changes when possibile (for example comreference) to minimize diff
5. now diff VSEXT and SHRPRJ should be minimal => phase two: review the code, and write a roadmap with small iterative changes

everything should be done in master, because using a long feature branch is worse (difficult to contribute, need sync, need merge, etc)

If the project fails (no one want to do it), at least we have reformatted code, and we remove the directory sharedprj ( the subtree is a normal directory )

[dsyme]

@enricosada I'm speaking for @KevinRansom and @NumberByColors and @otawfik-ms here (and I may be wrong)...

I think the reason for a lack of response here is because there is, at the moment, no definite, specific, certain roadmap for replacing or revamping the project system. It needs to be done, definitely. But it's not the first thing Microsoft are doing. Further, there are some forces pulling strongly to shard-project-system (MPF/SPS), and some strongly to CPS.

Microsoft are fully to be expected to be conservative here: they are on hook for every bug in the toolchain, to the point of making hotfixes in enterprise releases. I think we need to have Microsoft's full commitment to revamping the project system before it's really going to happen, but we can help them get there. I don't expect wholesale changes to be accepted until Microsoft are committed. In the meantime experimentaion, investigation, and selectively bring across functionality and fixes from MPF/SPS seem to be in order?

For a major revamp, using MPF/SPS is attractive, for the reasons you indicate. You summarize that well. However using SPS is also very attractive since it ultimately aligns with C# (as I understand it). And we all know that's where all the project system functionality gets implemented first. Aligning with C#'s implementation of the project system could give us a very smooth ride going forward. If Microsoft are ultimately going to do a major revamp, then the result will be very good, and well placed to add more and more functionality.

That said, I would love to see a "spike" of using MPF/SPS which could be road-tested to get a feel for where we are. Or even a rewrite of the whole project system using MPF/SPS. But exploring and assessing seems to be what's needed still, along with some tactical fixes and functionality.

This is all on behalf of the Visual F# Tools guys, and they may change/contradict/correct what I say freely :)

Don

p.s. This is more my own idea: I sometimes wonder if the Visual F# Power Tools should include an optional configuration choice which makes them act as a complete replacement for the Visual F# Tools (both Language Service and Project System) A full toolchain that could eventually be folded back here. We're all impressed with how much VFPT have implemented and I sometimes wonder why they don't go the whole way and (optionally) replace everything - and I suspect that would drive a lot of very cool things if they did.

[dsyme]

@enricosada Getting back to your plan, I do like the basic idea of minimizing the diff with SPS. But why do you need to integrate as a submodule to achieve steps 2, 3 and 4 above? Why not just submit the PRs without the submodule in the tree? I think we would accept basic, safe reformatting PRs that minimized the diff - at least, I don't see any problem with that.

[enricosada]

About why subtree

@dsyme it's not as submodule, it's as subtree. Subtree it's exactly like commit the code in a directory (but the commit has the info about source commit id, so you can pull/diff/sync easy)
I want to commit the sharedproj code inside the repo because it's easy to work with (not only for me).
Someone can clone the repo, diff the directories, and send pull requests. It's also easier for discussion.
Otherwise (without sharedproj source in repo), we need to tell people the exact commit of sharedproj to diff. if we need to update it, send fix back to sharedproj, it's more work.
Another good it's we can remove current files after are binary equals ( linking files inside SPS dir in projects ), so less codebase to diff

About roadmap

I think the reason for a lack of response here is because there is, at the moment, no definite, specific, certain roadmap for replacing or revamping the project system. It needs to be done, definitely. But it's not the first thing Microsoft are doing. Further, there are some forces pulling strongly to shard-project-system (MPF/SPS), and some strongly to CPS.

For sure it's not the priority, but the problem exists (really, install PTVS or nodejs and see the state of the art with sharedproj and projectsystem/azure/etc ).

Community can help, but need guidance about where to go.
If ms team cannot do it because of priority is coreclr (that's good and it's ok, no one is blaming ms visualfsharp team for anything), the community can help.
Asking for bugfix it's not working atm ( pls see issues/pr and people who tried )

Ask for small bugfix is another matter, but very few send bugfix to vs ide, that's the current situation, because it's lots of time, very little results and knowledge required is not documented ( it's not like fix a fsharp.core function, you need to know vside extensions, current hacks, code is C#/F#/VB etc )

About CPS vs SPS, ok, but what's the strategy? waiting two years who win the battle? CPS is going to win, but it's ready atm? also docs. SPS is working in two very good open source projects (and others like rust), so it's easier to steal and contribute back. atm.

Also CPS with c# mean .xproj atm (and project.json). i really think .xproj + project.json + dotnet/cli can fix all our problems (vside feature/files/azure/c# parity), but that's mean move to project.json.

It's really easy to see CPS in action in vs2015, create a c# project, rename from .csproj to .xproj ( ref Microsoft/VSProjectSystem/doc/scenario/turn_on_cps_for_csharp_vb_project.md . An example? see project properties.

I'll work for sure next week to make f# work in xproj, because it's going to help us a lot, but current working fsproj improvements? Also some stuff are shared, like intellisense.

Also, if both PTVS and nodejs are going to move to CPS, having the same codebase can help us to see how to migrate from current MPS to CPS. the c# project system is different, it's not MPS based i think (i can be wrong but i think it's c++ and a client to roslyn workspaces)

[dsyme]

About CPS vs SPS, ok, but what's the strategy?

I think the Visual F# Tools team will need to make a call on this eventually. I agree that SPS looks like a healthy option when viewed from the open source perspective. When viewed from Microsoft's position as long term maintainers and investors in new tooling, CPS may look better: ultimately the project system is part of Microsoft's product, supporting Microsoft's platform/business aims and giving a link to other platform-oriented VS tooling. And they need to consider the 2-10 year perspective.

We had a similar decision originally in 2007: do we take the C++ code for the C# project system and /s/CSharp/FSharp/g. Or do we use managed code MPF/SPS. Using C++ would have been immensely painful but would have given us alignment and more functionality. It was a devil of a choice and some of the problems with the Visual F# Tools project system stem from the fact we relied on the non-aligned choice.

I don't envy the decision. In the meantime I've given some advice above, which I repeat here (others may have different advice):

1. Help implement whatever project system support is going to be needed for CoreCLR. xproj, project.json, whatever.
2. Align. Help textually align the existing source with MPF/SPS in non-intrusive ways
3. Bug fixes and micro features. It is possible to do these, but it is hard. Aligning with MPF/SPS may help since some features from there may start to be more obvious.
4. Incremental Experiment. Consider doing a "spike" of replacing the existing C# source for the project system with either MPF/SPS, which could be road-tested to get a feel for where we are at
5. Radical experiment. Consider experimenting with a rewrite of the whole project system using MPF/SPS, probably as a contribution to the Visual F# Power Tools.

[dsyme]

Hey @enricosada - what's your skype address? Was wanting to ask you some questions about test suite runners.

[enricosada]

@dsyme my skype: enrico.sada

[NumberByColors]

I need to familiarize myself more with the project system space...so I don't have strong opinions, yet. We're (MS team) planning our work for 2016 over the next couple weeks. This would be what major projects we'd tackle after .NET Core support and support for the next version of Visual Studio are in good shape. Improving/replacing the project system is on the short list, but we need to decide how it ranks against improving the editor, debugging, and host of other things. We'll share as soon as we're in agreement with ourselves.