Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Obsolete] How to enable type providers with new-style .NET SDK project files, dotnet build, .NET Standard and .NET Core programming #3303

Closed
dsyme opened this issue Jul 4, 2017 · 89 comments
Labels

Comments

@dsyme
Copy link
Contributor

@dsyme dsyme commented Jul 4, 2017

This issue documents how to enable the use of type providers when using a .NET SDK 2.0 "new style" project file.

This also allows the use of cross-targeting type providers as part of .NET Core 2.0 app programming or .NET Standard 2.0 library programming.

If you don't follow one of the workarounds below you may get a silent failure or RunFsc.cmd failed from the F# compiler when executing dotnet build or "StackOverflowException", if your project references a type provider.

What you need to do

  1. Install an F# Compiler that runs on .NET Framework or Mono, see Linux, Windows, OSX. It is likely you already have one installed.

  2. Add the file fsc.props to the root of your project and import it into each of your project files that reference a type provider using something along the lines of: <Import Project="fsc.props" />,

The F# compiler will then run using .NET Framework and/or Mono.

Alternative Workaround

On Windows:

  • Install Visual Studio 2017 15.3 or later
  • Use "msbuild.exe" from a Visual Studio 2017 command line prompt to build the project instead of "dotnet build"

On Linux/OSX/Xamarin/Mono:

  • Install Mono 5.0 or later
  • Use "msbuild" to build the project instead of "dotnet build"

Explanation: when you compile your project using msbuild.exe the F# support in the .NET SDK will assume .NET Framework execution of the F# compiler is required. This workaround is also compatible with using an fsc.props file as mentioned above.

Examples where the problem occurs

Example 1 (building a .NET 4.6.1 application that uses FSharp.Data)

dotnet new console -o -lang F# --target-framework-override net461 app3
cd app3
dotnet add package FSharp.Data
dotnet restore
dotnet build

Example 2 (building a .NET Core 2.0 application that uses FSharp.Data)

dotnet new console -o -lang F# app4
cd app4
dotnet add package FSharp.Data
dotnet restore
dotnet build

Expected behaviour is compilation success, or at least nice error point to workaround. Actual behavior is a silent compilation failure, and dotnet build /v:d shows l Process is terminating due to StackOverflowException. Done executing task "Fsc" -- FAILED.

Explanation

As explained in this issue, type providers are not yet supported when the F# compiler runs using .NET Core. By default, the .NET SDK tooling runs the F# compiler using .NET Core.

The workaround is to have the .NET SDK tooling use an F# Compiler running on the .NET Framework or Mono, regardless of which version of .NET you're targeting. By doing this, the type provider is hosted and executed using the .NET Framework. This technique applies even if you are doing .NET Core programming (for cross-targeting type providers), or building a .NET Standard 2.0 library. The F# Compiler cross-generates code for the correct target regardless of how it executes.

Although this workaround may seem "wrong" if targeting .NET Core, it has advantages:

  • the use of .NET Framework/Mono is only at compile-time
  • the technique stress-tests the cross-targeting capabilities of the type provider
  • the technique unblocks TP authors to test and adjust TPs to be cross-targeting

In the future, the plan of record is to allow .NET Standard 2.0 type providers to be used when the F# compiler is running on .NET Core. This is an interim workaround for those who wish to unblock the testing and use of type providers for .NET SDK project files and .NET Core/.NET Standard 2.0 programming.

@dsyme dsyme changed the title StackOverflow when F# compiler in .NET SDK references FSHarp.Data.TypeProviders StackOverflow when F# compiler in .NET SDK references FSharp.Data.TypeProviders or FSharp.Data Jul 4, 2017
@dsyme dsyme changed the title StackOverflow when F# compiler in .NET SDK references FSharp.Data.TypeProviders or FSharp.Data StackOverflow when F# compiler in .NET SDK 2.0 preview 2 references FSharp.Data.TypeProviders or FSharp.Data Jul 4, 2017
@dsyme dsyme changed the title StackOverflow when F# compiler in .NET SDK 2.0 preview 2 references FSharp.Data.TypeProviders or FSharp.Data StackOverflow when F# compiler in .NET SDK 2.0-style project file references a type provider - must use "msbuild"/"xbuild" NOT "dotnet" to build Jul 4, 2017
@dsyme dsyme changed the title StackOverflow when F# compiler in .NET SDK 2.0-style project file references a type provider - must use "msbuild"/"xbuild" NOT "dotnet" to build When using .NET SDK 2.0-style project file and you reference a type provider you must use "msbuild"/"xbuild" NOT "dotnet" to build Jul 4, 2017
@dsyme dsyme changed the title When using .NET SDK 2.0-style project file and you reference a type provider you must use "msbuild"/"xbuild" NOT "dotnet" to build When using .NET SDK 2.0-style project file and you reference a type provider you must use "msbuild" NOT "dotnet" to build Jul 4, 2017
@dsyme
Copy link
Contributor Author

@dsyme dsyme commented Jul 4, 2017

Note, I think we can have a better setting for FscToolPath which works on both Windows and Mono, though it needs a conditional to determine the right location

@dsyme dsyme changed the title When using .NET SDK 2.0-style project file and you reference a type provider you must use "msbuild" NOT "dotnet" to build If a .NET SDK 2.0-style project file references a type provider you must use "msbuild" NOT "dotnet" to build Jul 4, 2017
@dsyme dsyme changed the title If a .NET SDK 2.0-style project file references a type provider you must use "msbuild" NOT "dotnet" to build If a .NET SDK 2.0-style project file references a type provider you must use "msbuild" NOT "dotnet build" to build Jul 4, 2017
@larjo
Copy link

@larjo larjo commented Jul 4, 2017

Thanks @dsyme!

The FscToolPath/FscToolExe workaround seems to work just fine even with SDK 1.0.4.

@dsyme
Copy link
Contributor Author

@dsyme dsyme commented Jul 4, 2017

@cartermp @KevinRansom As I've now documented in the issue description, it is possible to use the above technique to enable type providers for .NET Core programming, as long as the type providers have been written as a cross targeting type provider. The main cross-targeting type provider is FSharp.Data.

Here are the steps I used to verify that this can be used with .NET Core 2.0 programming:

  • dotnet new console -o -lang F# tpapp2
  • cd tpapp2
  • dotnet add package FSharp.Data
  • dotnet restore
  • edit the .fs to contain
open System
open FSharp.Data

type NugetStats = HtmlProvider<"https://www.nuget.org/packages/FSharp.Data">

let rawStats = NugetStats().Tables.``Version History``

rawStats.Rows |> Seq.iter (printfn "row = %A")
  • edit the tpapp2.fsproj to contain
    <FscToolPath>C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0</FscToolPath>
    <FscToolExe>fsc.exe</FscToolExe>
  • dotnet build
  • dotnet run

output:

row = ("F# Data 2.3.3 (this version)", 22954M, 10/04/2017 00:00:00)
...
row = ("FSharp.Data 1.0.2", 234M, 14/12/2012 00:00:00)
row = ("FSharp.Data 1.0.1", 187M, 14/12/2012 00:00:00)
row = ("FSharp.Data 1.0.0", 469M, 13/12/2012 00:00:00)

🎉🎉🎉🎉 we have used type providers for .NET Core programming for the first time :)

Notes:

  • When using VSCode/Ionide you also get full intellisense:

  • The above technique also works for .NET Standard 2.0 programming (dotnet new lib -lang F# tplib1). However at the time of writing you need an F# compiler that includes this fix, e.g. use Visual Studio 2017 preview 3, or use an F# compiler built from source. I tested with <FscToolPath>C:\GitHub\dsyme\visualfsharp\release\net40\bin</FscToolPath>

  • You get some warnings because FSharp.Data for .NET 4.x is being used. However it should be possible to produce a fully compliant .NET Standard 2.0 version of FSharp.Data, that is TBD, and until then the .NET Framework version is not that likely to cause major problems since the surface area of FSharp.Dataa is relatively low

image

@dsyme dsyme changed the title If a .NET SDK 2.0-style project file references a type provider you must use "msbuild" NOT "dotnet build" to build How to enable type providers with new-style .NET SDK project files Jul 4, 2017
@dsyme dsyme changed the title How to enable type providers with new-style .NET SDK project files How to enable type providers with new-style .NET SDK project files (and in some cases .NET Core programming) Jul 5, 2017
@dsyme dsyme changed the title How to enable type providers with new-style .NET SDK project files (and in some cases .NET Core programming) How to enable type providers with new-style .NET SDK project files (and in some cases .NET Core programming too) Jul 5, 2017
@buvinghausen
Copy link

@buvinghausen buvinghausen commented Jul 7, 2017

@dsyme you are the freaking man! If I'm ever at a conference you're speaking at dinner one night is on me...

So I journeyed into the matrix to get my application running inside a Docker container on Linux despite it's heavy JSON Type Provider usage due to your workaround and I'm happy to report it works like a champion!

Pros:
This workaround is elegant, nice, and awesome
Runs in Docker on Linux!!!!!
Only developer IDEs and build servers need Windows which is perfectly livable until type providers compile on Core
I can still write all my build scripts for the build servers to be dotnet cli compliant

Cons:
There were 3 things that came up and I've attached a screenshot for all of them to see if I'm doing something wrong..

  1. FSC was complaining about netstandard2.0 but it works when I put in netcoreapp2.0
    fsc_netstandard_error

  2. NuGet shows no packages installed for F#
    nuget_shows_nopkgs

  3. My Intellisense is broken but I'm pretty certain this probably has something to do with item 2
    vs_errors_netcore2

The only other con was dotnet build, run, restore, publish all throw errors when I include the docker-compose.dcproj file in the .sln file. I have read that is because the SDK hasn't been open sourced which is a requirement to be included in the tool chain. This is not a concern as I can simply script the commands to all run against the top level project file rather than the solution.

I have a feeling my cons are entirely a PEBCAK and I've thought up workarounds in the event they are not. I'm still way happier about actually being able to build out my services against dockerswarm on Linux and still being able to use the Type Providers.

Regards,
Brian

@buvinghausen
Copy link

@buvinghausen buvinghausen commented Jul 7, 2017

The PEBCAK is that I couldn't read.... vscode/ionide I will leave the items up here just in case we want the VS team to see they are very close.

@cartermp
Copy link
Contributor

@cartermp cartermp commented Jul 7, 2017

Not sure about (2), but (3) is due to an issue in the language service. #3260 is tracking that issue, and @KevinRansom is currently working on it. This is wonderful to hear, though, @buvinghausen! Happy to hear that it's working for you in Docker/Linux.

@enricosada
Copy link
Contributor

@enricosada enricosada commented Jul 7, 2017

Only developer IDEs and build servers need Windows which is perfectly livable until type providers compile on Core

Atm will work also on non windows, using mono. two big ways:

  1. use mono on dev on non win. same workaround

  2. use docker multi stage setup

    • build a mono docker image for build, build there => get output
    • run run the build output in another container with just .net core

pro of 2 is that with multi stage, the image size contains only netcore (use official dotnet core docker image) and is slimmer

maybe i should gist the dockerfile i use for that. is a good setup, also for ci who are docker based (we use jenkins).

@enricosada
Copy link
Contributor

@enricosada enricosada commented Jul 7, 2017

@dsyme as a note, dotnet new lib -lang F# works the same of classlib, and is less wierd 😄

@buvinghausen
Copy link

@buvinghausen buvinghausen commented Jul 7, 2017

@cartermp thanks for the clarification I'm actually working on getting VS Code launch & debug into the Docker container like I can from VS 2017 then I may just cut the cord on VS full I've been looking for an excuse to do that as I have been using VS Code exclusively for TypeScript development.

@enricosada Those are some excellent points but for now I'm totally happy with simply making my CI server run on Windows and letting the FE developers run everything in Node but talking to our integration services on the back end. That should give me enough runway to let all the frameworks and tooling catch up such that I can then tell them they have to install .NET Core & OmniSharp and learn the dotnet cli. As an aside I think today is the day that I'm going to see how feasible it is to run the React Redux template in Docker. I know when you create the project they have the docker option disabled. Should be fun.

I really appreciate all the work you guys are putting into this platform I cannot stress that enough it really is amazing to think what the future holds.

@orient-man
Copy link

@orient-man orient-man commented Jul 18, 2017

Thx for the workaround but I encountered another issue:

error FS3033: The type provider 'FSharp.Data.TypeProviders.DesignTime.DataProviders' reported an error: The .NET SDK 4.0 or 4.5 tools could not be found

Source code tells the truth: SDK paths are fixed. On my relatively new machine (or on VSTS agent) there're no such registry keys. I added new ones (which existed on my machine):

@"Software\Microsoft\Microsoft SDKs\NETFXSDK\4.6.1\WinSDK-NetFx40Tools"
@"Software\Microsoft\Microsoft SDKs\NETFXSDK\4.6.2\WinSDK-NetFx40Tools"

and now it works. Are you interested in pull request for that change?

@dmitry-a-morozov
Copy link

@dmitry-a-morozov dmitry-a-morozov commented Aug 1, 2018

Latest commit de10d57 has workaround commented out
https://github.com/demetrixbio/FSharp.Data.Npgsql/blob/de10d5749dbdf3d8dfb9292bcaaa245f8da9629b/tests/Tests.fsproj#L10

So it compiles from VS but fails from command line.
I actually wanted to take it step father and place TPDTC in
...\typeproviders\fsharp41\netstandard2.0\FSharp.Data.Npgsql.DesignTime.dll relative to runtime component but legacy location (design time co-located to run-time) doesn't work either.

Follow instructions to run tests.

Honestly I think it's better to extend BasicProvider example to have dependency because it will useful for community as example of managing dependencies. But I'll leave it to your judgement.

@dmitry-a-morozov
Copy link

@dmitry-a-morozov dmitry-a-morozov commented Aug 1, 2018

@KevinRansom I made some changes to unit test suite so it should run on your machine.
Use this commit
demetrixbio/FSharp.Data.Npgsql@b44df12

Thanks

@boeremak
Copy link

@boeremak boeremak commented Aug 7, 2018

@buvinghausen Thanks for the tip on FSharp.Data beta, it also fixed my issue.

Still a brave new world this Dotnet core. :)

@dmitry-a-morozov
Copy link

@dmitry-a-morozov dmitry-a-morozov commented Aug 17, 2018

@KevinRansom
Congrats on 15.8 VS release ! Great work.
Do you have any updates on discussion above and fsprojects/FSharp.TypeProviders.SDK#244?

Thanks

//cc @dsyme @cartermp

@KevinRansom
Copy link
Member

@KevinRansom KevinRansom commented Aug 17, 2018

@dmitry-a-morozov It's on my list of things to resolve. I'm afraid I have no details about when :-(

@dmitry-a-morozov
Copy link

@dmitry-a-morozov dmitry-a-morozov commented Aug 17, 2018

@KevinRansom I understand. Keep me posted.

@ctaggart
Copy link
Contributor

@ctaggart ctaggart commented Oct 11, 2018

I so confused on what I need to do to get the OpenAPITypeProvider to work. It is build for netstandard2.0. fsprojects/OpenAPITypeProvider#6

@dsyme
Copy link
Contributor Author

@dsyme dsyme commented Oct 12, 2018

Recording a link to my reply here: fsprojects/OpenAPITypeProvider#6 (comment)

@SpiegelSoft
Copy link

@SpiegelSoft SpiegelSoft commented Feb 2, 2019

I'm trying to use FSharp.Configuration in my project. On my TeamCity build environment (Linux), I have installed the fsharp compiler and imported fsc.props, but the build breaks with the error message

"fsharpc" exited with code 1.

For now, I'm not sure where to go, as it's not clear how to get a detailed error description from fsharpc.

@larjo
Copy link

@larjo larjo commented Apr 23, 2019

The fsc.props workaround stopped working in version 2.2.203 of the SDK.

I posted a workaround in fsprojects/FSharp.TypeProviders.SDK/issues/244

@buvinghausen
Copy link

@buvinghausen buvinghausen commented May 6, 2019

Why would you still be using the fsc.props workaround? The 3.0 beta (and all subsequent 3.x releases) has been working without the workaround for over a year now...

@Thorium
Copy link
Contributor

@Thorium Thorium commented May 29, 2019

After the latest caching improvements I started to get this error FS3033(0,0): The design-time type 'System.String' utilized by a type provider was not found in the target reference assembly set '[ ...(list of assemblies)... ]'. You may be referencing a profile which contains fewer types than those needed by the type provider you are using. also in the .NET Framework full.

ProvidedTypes.fs commit 18459e8b23b44b389dc8f2b8fe444fedf2b47a8c did work, but after commit b08fe13625b2ffc08529cb9350db8f86c6b3c537 there are problems.

@patham9
Copy link

@patham9 patham9 commented Jun 7, 2019

It's fixed in 3.0? With dotnet-sdk-3.0.100-preview5-011568-linux-x64.tar.gz I still get the same error, after invoking "dotnet build" on a F# project:
error FS3053 : The type provider 'FSharp.Configuration.ConfigTypeProvider+FSharpConfigurationProvider' reported an error : The type provider constructor has thrown an exception: Exception has been thrown by the target of an invocation.
Will it be fixed properly, if so when is a fix planned to appear approximately? Can a out-of-the-box solution that does not demand manual workarounds with path hacks be expected in the future?

@cartermp
Copy link
Contributor

@cartermp cartermp commented Jun 7, 2019

@patham9 it’s highly unlikely that this issue is the right place to report that error, as the mechanism to allow type providers to work on .NET Core has been shipped and stable for quite some time. I suspect your issue lies in the library, not this type provider mechanism.

@baronfel
Copy link
Contributor

@baronfel baronfel commented Jun 7, 2019

I agree with @cartermp. My initial porting of the FSharp.Configuration TP was incomplete and it hasn't been fully updated with a .netstandard version, which would be required for seamless usage. There's an issue on the repo for this I believe, but someone needs to do/finish the work. It's on my list to do, but like many others I have many competing priorities.

@jfrank-summit
Copy link

@jfrank-summit jfrank-summit commented Jun 28, 2019

@Thorium Did you ever figure out a workaround or fix? Having this same problem...

After the latest caching improvements I started to get this error FS3033(0,0): The design-time type 'System.String' utilized by a type provider was not found in the target reference assembly set '[ ...(list of assemblies)... ]'. You may be referencing a profile which contains fewer types than those needed by the type provider you are using. also in the .NET Framework full.

@cartermp cartermp changed the title How to enable type providers with new-style .NET SDK project files, dotnet build, .NET Standard and .NET Core programming [Obsolete] How to enable type providers with new-style .NET SDK project files, dotnet build, .NET Standard and .NET Core programming Jun 28, 2019
@Thorium
Copy link
Contributor

@Thorium Thorium commented Jun 28, 2019

Yes, I figured out what happened.
FSharp.TypeProviders.SDK has the file ProvidedTypes.fs which is loading types from netstandard.dll which contains the base types of .NET Standard.
If you follow the tutorial, LemonadeProvider, and open the file
src\LemonadeProvider.DesignTime\LemonadeProvider.DesignTime.fsproj you will notice the dependency:

    <None Include="..\..\packages\NETStandard.Library.NETFramework\build\net461\lib\netstandard.dll" Condition="'$(TargetFramework)' == 'net45'">
        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>

So the typeprovider loads the netstandard.dll from this NuGet package:
https://www.nuget.org/packages/NETStandard.Library.NETFramework/2.0.0-preview2-25405-01

But the netstandard.dll there is an old version, and if you a later use netstandard.dll in your machine (you have the latest .NET Core installed) there will be version conflict between the loaded netstandard.dll and the used netstandard.dll. That might also lead the type-provider to reflection-loading-loop between the two dlls, causing stackoverflow, or just not to find the core types. Using only the old-one works.

I went the simplest fix which was uninstall VS2019 and take VS2017 back, and everything worked correctly.
:-P

Edit: Also, the ProvidedTypes.fs I'm using is the commit 18459e8b23b44b389dc8f2b8fe444fedf2b47a8c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.