2017 Status and Roadmap for F# and the Visual F# Tools #2400

Open
cartermp opened this Issue Feb 9, 2017 · 25 comments

Projects

None yet
@cartermp
Contributor
cartermp commented Feb 9, 2017 edited

Current Status of F# and the Visual F# Tools per VS2017 RTM

  • F# 4.1 is feature complete
    • Multiple new features, including struct representation of types, a new Result type, and other areas that are “nip and tuck” in nature
    • Multiple bug fixes
    • One by-design breaking change for C#-style Extension Methods
    • One known regression (#2373)
    • One known bug in multi-case Struct DUs (#1678)
  • .NET Framework compiler in good shape
  • .NET Framework FSI in good shape
  • .NET Core compiler in RC
    • Type Providers and Codegen from Quotations are not supported due to changes made in reflection APIs for .NET Core. These will be in .NET Core 2.0+ timeframe.
  • .NET Core FSI works as a simple REPL
    • However, it does not support the .NET Core references model and thus does not support #r
  • Visual F# Tools in Visual Studio 2017 is sitting atop Roslyn Workspaces with many new features thanks to the OSS Community
    • It's much easier to add new features to the tooling now
    • However, there are numerous regressions that are going to be fixed
  • Visual FSI window in Visual Studio 2017 is in good shape
  • Debugging is in good shape, with multiple areas of improvement
    • Support for Portable PDBs and Desktop PDBs on .NET Framework
    • Support for Portable PDBs on .NET Core
    • Support for Embedded Portable PDBs on .NET Framework and .NET Core
    • Support for embedded source code with Portable PDBs on .NET Framework and .NET Core
    • Points of significance:
      • Still no Expression Evaluator (EE) for F#, so that means many debugging features are still unavailable as they always have been
      • Regression when debugging F# Lists: the .Items property is not invoked by the Roslyn EE used in Visual Studio
      • No other regressions that we can find
      • Legacy EE will be going away
  • The Visual F# Tools project system does not support F# on .NET Core
    • The new .fsproj file does not load in VS
    • No project system support for package load, cross-platform build, or publish
    • No .NET Core templates in-box
    • This means you can create something with the .NET CLI and have it fail to load in Visual Studio
  • Lightweight Solution Load is not compatible with F# for VS 2017 release.
    • Build/Rebuild/Clean do not work
    • Text searching through any F# projects do not work either.

Immediate Roadmap for 2017

  1. Bug fixes

    • Fixing tooling regressions in VS 2017 tooling
    • Fixing serious bugs in new features in VS 2017
    • Fixing serious bugs in the compiler
  2. Stability

    • It’s crucially important that we get our tooling stability up to spec. Many new features were introduced, but we didn’t increase our test coverage by much.
    • There is currently a bug on our side which prevents the Connected Services node in VS 2017 from working with F# projects.
    • Performance and memory-consumption issues
      • There were regressions here introduced in VS 2017 compared with VS 2015. It will be an ongoing battle to improve this, but we’ve already seen big improvements from Don just recently.
    • Build/Rebuild/Clean and Text Search do not work if Lightweight Solution Load is turned on. We’ll work with the team that owns this to get this fixed.
  3. Migrate the OSS Build to the latest .NET CLI and MSBuild and use the Multilingual App Toolkit (MAT) for localization builds

    • Migrating OSS Build to latest .NET CLI and MSBuild is necessary for .NET Core 2.0 support, which is in turn necessary to get Type Providers, Quotation support, and #r support in F# for .NET Core
    • Using the MAT will also bring down our internal build times for signed binaries tremendously
  4. Support generating signed VSIXs of the latest master build, pushed directly to MyGet

    • This will enable people to get on a “Fast track” for F# tooling in VS 2017, test out brand-new features, and provide feedback early on
    • The artifacts produced will come with the assumption that people know they won’t be as well-tested and may introduce regressions
  5. Project System support for .NET Core and .NET Standard projects in F#

    • Ability to load MSBuild 15-based .fsproj projects
    • Add gestures for cross-platform builds, publish, and adding package references
    • Consideration: Roslyn Project System
      • Add F# support to the new project system that C# and VB use for .NET Core/Standard projects
      • Remove our own project system entirely, making us “first-class” in that regard
      • This is pending an investigation into the work involved
  6. Full FSI support on .NET Core 2.0

    • Properly handle the model for how .dlls are laid out on disk with .NET Core, thus fixing #r
    • Establish a good working model for how to do the above
    • Prompt to re-load an FSI session when #r’ing an assembly that targets a higher target than what FSI was launched with?
      • Error message asking user to re-load FSI manually?
      • Try to re-target and re-reference everything on the fly?
  7. Full support for F# on .NET Core 2.0 and .NET Standard 2.0

    • Type Providers and Code Quotations
    • This also takes a dependency on RefEmit. We’ll need to investigate this further, since it’s currently not on the .NET Core 2.0 roadmap
  8. Build a Contract .dll between Roslyn and F# to remove our use of InternalsVisibleTo

    • This will allow Roslyn to determine when they break us if they make any breaking changes to their APIs
    • This will be a big improvement over the current situation, where we must manually check this
  9. OSS Engineering Efficiency and Build needs some improvements

    • The .NET CLI does not use project.json anymore, but our OSS Build still relies on that
    • The .NET CLI uses MSBuild 15 and PackageReferences, which we need to support
    • Open Source Localization is needed to simplify our localization process and more easy leverage the OSS community
      • Using the Multilingual App Toolkit is a way to improve our internal process and reduce build times

In accordance with the above, what follows is where we expect work will come from. In the case of the OSS Community, while we don't assume work will be done for us, we anticipate continuing to receive high-quality contributions which we will gladly accept.

@KevinRansom

  • VS support for loading .NET Core 1.x projects in Visual Studio
    • This is an issue being addressed in the short-term
  • F# Compiler support for .NET Core 2.0
  • FSI support for .NET Core 2.0
  • Type Provider and Code Quotations support for .NET Standard 2.0
  • System.ValueTuple work to deal with its migration to mscorlib in .NET Framework 4.6.3
  • Helping with concrete work in evolving F# 4.2
  • Bug fixes in the compiler, PDBs, debugging, and platform support

@brettfo

  • Bug fixes in the VS Tooling
    • Addressing regressions and serious bugs in new features
    • Connected Services Node bug and other issues with VS features
  • Addressing the OSS Build
  • Getting our internal builds to a place where we can publish signed VSIXs for people to use
  • Addressing Localization
    • Using the Multilingual App Toolkit to reduce build times and remove our internal clone of this repo
    • Improve OSS Localization
  • Investigating F# support on the Roslyn Project system, and acting on that if it's deemed reasonable
  • Support the community

@cartermp

  • Blog post with a focus on OSS attribution for F# 4.1 and the Visual F# Tools in VS 2017
  • Outreach to internal teams to address issues that affect us
  • Support the community
  • Low-impact bug fixes, if possible

@dsyme and the F# OSS Community

  • F# 4.2 evolution
  • Additional IDE features and improvements
  • Performance improvements
  • Bug fixes (note: this does not mean that we are punting bug fixes to the community. It means we will accept any bug fixes from the community)

Roslyn Team

  • Helping us and the OSS community better understand Roslyn when working on features
  • Engaging with and helping unblock the OSS community
@GiorgioG
GiorgioG commented Feb 9, 2017

I'm surprised the project system support for .NET Core / MSBuild 15-based .fsproj projects is so far down on the list. Given that .NET Core presents an opportunity to use F# (in part at least) for greenfield/rewrite projects, I would have thought a higher priority would have been given to this since it could help increase adoption numbers.

@cartermp
Contributor
cartermp commented Feb 9, 2017

@GiorgioG Supporting .NET Core and MSBuild 15-based projects is the top item for @KevinRansom. I wouldn't interpret numbers in the list as an overall priority.

@smoothdeveloper
Contributor

@GiorgioG I'm not sure if the order in Immediate Roadmap for 2017 is indication (localization and build optimization are high but they don't seem to be the most blocking) but I see it is first item @KevinRansom has on the detailed list.

@charlesroddie

Why isn't Windows Store support (related to .NET Native support) on this list?

@KevinRansom
Contributor

@charlesroddie Because that depends on the Core CLR team improving support for a couple of big items. dotnet native is and AOT compilation are absolutely a goal for us ... they just are longer term than this roadmap anticipates.

I hope this helps

Kevin

@KevinRansom
Contributor

@georigiog @smoothdeveloper I am looking at it right now ... I am in fact torn about how to proceed.

I would like to go CPS since that is the long term right plan ... however ... CPS fundamentally orders files alphabetically ... this has been a long known bug in CPS and is still not fixed. It has not been fixed because .. .csproj and .vbproj don't need it, and .fsproj doesn't load into CPS. It's been kind of a chicken and egg problem. Also CPS isn't OSS so fixing it's behavior requires somewhat more ceremony than we are used to.
Also would require that we continue to ship the old project system for legacy projects.

An alternative approach and in the short term less risky approach is to fix the F# project system to load them. However ... there are a bunch of nice but large work items that we would then not have.

  1. Built in support for Roslyn multi-targeting, especially intellisense
    (API narrowing --- select two target platforms and get the sommon subset of APIs).
  2. Support for package references

However ... file ordering will work ....

Anyway I am torn ....

Probably an overshare .... please forgive me if it was too much info ...

Kevin

@smoothdeveloper
Contributor
smoothdeveloper commented Feb 9, 2017 edited

@KevinRansom thanks, it is good to know what are the ongoing struggles.

It seems most people are looking for a quick fix to get .net core fsproj to load in VS, we all understand going toward CPS is a lot more work and will bring a lot to the table.

Getting out of box VS support for F# .net core projects is going to help with adoption of both .net core and F# for xplat (for windows/VS users), this is a quick win if we roughly what needs to be dealt with in current project system.

Edit: should the discussion regarding that focused item go to #2394 ?

@GiorgioG
GiorgioG commented Feb 9, 2017

@KevinRansom Thanks for the details Kevin - it's helpful to understand that there are complex issues / tradeoffs / decisions to be considered.

@enricosada
Contributor

Migrate the OSS Build to the latest .NET CLI and MSBuild and
The .NET CLI does not use project.json anymore, but our OSS Build still relies on that
The .NET CLI uses MSBuild 15 and PackageReferences, which we need to support

I'd like to do that.
Speaking with @cartermp , we can discuss (in another issue) how to that with @brettfo and community.
I already have a PR about that. Need to just complete it building also fsharp.core/fsharp.compiler, and replacing build coreclr for now, as @dsyme suggested
Signing/localization too, can be done like dotnet/cli team (ref .xlf localization files really nice ihmo so we can contribute to localization too!, like other ms projects)

@KevinRansom
Contributor

@enricosada ... reorganizing the build is about lot more than just changing the way we build the coreclr. The current build is absolutely horrific, and we should take a stab at fixing it all up.

@enricosada
Contributor
enricosada commented Feb 9, 2017 edited

@KevinRansom i want to do that too (xplat, no hacks, from clone -> signed packages + vsix, like dotnet/cli does for example). But incrementally, first coreclr so we can clean and use sdk and unblock netstandard2,0 if needed, After that, the rest. with sdk i can remove pretty really a lot, see my PR #2250 (comment) about cleanup and future work.

I have some ideas, we can discuss goals and how to proceed in another issue maybe? i dont want to monopolize this issue 😄 i am always a lot verbose

@KevinRansom
Contributor

@enricosada I will take another look and suggest an approach.

@Huqin-China

Thanks for your work and achievements!
F# is a nice language,it will become perfect one after VS's full support.
VS is called "the most powerful IDE in Universe" in China.
Nice language+Great IDE,The Gospel of dummy Programmers

@dbettin
dbettin commented Feb 10, 2017 edited

This also takes a dependency on RefEmit. We’ll need to investigate this further, since it’s currently not on the .NET Core 2.0 roadmap

How high priority is this issue? .Net core engenders growth in F#, but we can't do it without type providers.

@cartermp
Contributor

@dbettin The breakdown is actually as follows:

Erasing Type Providers (which are more common) should not require System.Reflection.Emit, but Generative Type Providers will. We'll first need to ensure that with the .NET Standard 2.0 APIs coming along, that we can at least support Erasing Type Providers. If that's the case, then many of the type providers in the wild could conceivably be used, assuming there are no other issues we run into.

For Quotations, it's a somewhat similar story. They don't require System.Reflection.Emit, but they are needed for fully-fledged quotation evaluators/compilers, and are needed if they are used as input to a Generative Type Provider.

What I think may be likely is that we'll have support for the "95% case" in these areas, but we could still be blocking key libraries and library authors if they are building full-on quotation compilers and Generative Type Providers.

All of the above is assuming that we don't do any work in those areas ourselves. @KevinRansom can speak more towards the feasibility of that, but given the amount of work we're doing right now, I don't find it likely that rewrites of those areas of TPs and Quotations will come from someone in Redmond between now and .NET Standard/.NET Core 2.0.

@dbettin
dbettin commented Feb 10, 2017

Thanks @cartermp. It might be helpful to break these out into separate issues. And maybe assign an up for grabs label to issues that are not assigned to Redmond devs?

@dsyme
Contributor
dsyme commented Feb 10, 2017

@Huqin-China Thanks for the perspectives from China!

@KevinRansom
Contributor

@dbettin @cartermp

The problem with generative TypeProviders is that ref emit on coreclr doesn't implement Save(). Ref emit is not actually a part of .Net Standard 2.0 the System.Reflection.Emit package will work with fine with .Net Standard 2.0, it's just those Apis are not part of the .Net Standard set of Apis and so ... there is no urgency to improve it's desktop compatibility. The problem for F# is that generative type providers need to produce a P.E file and since the the Save() API doesn't exist, we need a clever solution to that problem. There are in fact plenty of approaches we could take ... we just haven't settled on one that is the obvious winner.

I cornered Jan Kotas in the corridor the other day and tried to push him to commit to getting us Save, he urged us to seek other solutions ...

So ... we need an alternate solution, regardless until .Net Standard 2.0 we can't do much with type providers given that Type has a private constructor on the coreclr, at least in the reference assemblies it is, the implementation still has a protected constructor. So I suppose we could do type reference magic to get something working.

As for up for grabs ... just express an interest in an area you want to work on and start a discussion. Helping out is really very easy. Most people just pick something they want to do and start work on it.

Kevin

@dbettin
dbettin commented Feb 10, 2017

@KevinRansom Thanks. My point around the up for grabs is it would beneficial to break these up so we can start tracking these issues independently.

@smoothdeveloper
Contributor

@dbettin do you mind creating the few issues you have in mind? I think with github option, the maintainers can expand the description with additional technical and contextual information.

@panesofglass

This roadmap makes it appear that, for my purposes, VS 2017 RTM is really more of an alpha release for F# support, at least wrt .NET Core. Will .NET 4.6 projects continue to work, or should we stick to VS 2015 and/or VS Code?

@cartermp
Contributor

.NET Framework 4.6 projects should continue to work, yes. Addressing any regressions there is a top priority for us. RC4 is a good target to validate this on. The bits in there are the bits which are shipping in VS 2017 RTM, barring some unexpected, massive error on our part.

@OnurGumus

Not a single word about .NET Native or UWP support. Nice!

@OnurGumus

@vasily-kirichenko ah ok thanks. Didn't read all the comments.

@dbettin dbettin referenced this issue in dustinmoris/AspNetCore.Lambda Feb 16, 2017
Closed

Migrate to .NET-Core-SDK-RC3 #9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment