FSharp.Compiler.Service: TypeInitializationException in GetProjectOptionsFromScript in published console application

[MortalFlesh]

Hello,

The bug was originally here fsharp/fsharp-compiler-docs#965

I have a project which uses a FSharp.Compiler.Service to parse .fsx files as a Domain specification and then uses results to check whether a use cases are correct (https://github.com/TypedUseCase/).

Recently I've came across this bug (not sure when it happened and why - I changed a lot after a last successful run - dotnet core, dependencies, ...).

Problem is in a self-contained published binary (after dotnet publish) - if I run a code by just dotnet run it works fine.

In a published code run I get this error on GetProjectOptionsFromScript call:

System.TypeInitializationException: The type initializer for '<StartupCode$FSharp-Compiler-Service>.$FSharpCheckerResults' threw an exception.
 ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at <StartupCode$FSharp-Compiler-Service>.$FSharpCheckerResults..cctor()
   --- End of inner exception stack trace ---
   at FSharp.Compiler.CodeAnalysis.FSharpCheckerResultsSettings.get_defaultFSharpBinariesDir()
   at <StartupCode$FSharp-Compiler-Service>.$Service.e@822-10(BackgroundCompiler _, FSharpOption`1 useSdkRefs, FSharpOption`1 useFsiAuxLib, ISourceText sourceText, FSharpOption`1 sdkDirOverride, FSharpOption`1 previewEnabled, FSharpOption`1 otherFlags, FSharpOption`1 optionsStamp, FSharpOption`1 loadedTimeStamp, String filename, FSharpOption`1 assumeDotNetFramework, CompilationThreadToken ctok, ErrorScope _arg36)
   at <StartupCode$FSharp-Compiler-Service>.$Service.f@826-139(BackgroundCompiler _, FSharpOption`1 useSdkRefs, FSharpOption`1 useFsiAuxLib, ISourceText sourceText, FSharpOption`1 sdkDirOverride, FSharpOption`1 previewEnabled, FSharpOption`1 otherFlags, FSharpOption`1 optionsStamp, FSharpOption`1 loadedTimeStamp, String filename, FSharpOption`1 assumeDotNetFramework, CompilationThreadToken ctok, Unit unitVar)
   at <StartupCode$FSharp-Compiler-Service>.$Service.GetProjectOptionsFromScript@871-1.Invoke(CancellationToken ct)
   at <StartupCode$FSharp-Compiler-Service>.$Reactor.EnqueueAndAwaitOpAsync@220-2.Invoke(CompilationThreadToken ctok)
--- End of stack trace from previous location ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit()
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInAnotherThread[a](CancellationToken token, FSharpAsync`1 computation, FSharpOption`1 timeout)
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout)
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
   at Program.DomainResolverLibrary.parseAndCheck(FSharpChecker checker, String file, String contents)

Repro steps

I've created a simple repo with a minimal code, which shows the error.
https://github.com/MortalFlesh/fscs-bug-example

There are also github actions with the runtime example:
https://github.com/MortalFlesh/fscs-bug-example/actions/runs/1127856732

I've described more details in the repo readme - I can copy it here, if it is needed.

Expected behavior

It should return a Project options as normal.

Actual behavior

In published code, it fails on The type initializer for '<StartupCode$FSharp-Compiler-Service>.$FSharpCheckerResults' threw an exception.

Known workarounds

I don't know any workarounds.

Related information

.NET SDK (reflecting any global.json):
 Version:   5.0.400
 Commit:    d61950f9bf

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.0
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   /usr/local/share/dotnet/sdk/5.0.400/

Host (useful for support):
  Version: 6.0.0-preview.4.21253.7
  Commit:  bfd6048a60

.NET SDKs installed:
  5.0.103 [/usr/local/share/dotnet/sdk]
  5.0.200 [/usr/local/share/dotnet/sdk]
  5.0.300 [/usr/local/share/dotnet/sdk]
  5.0.301 [/usr/local/share/dotnet/sdk]
  5.0.302 [/usr/local/share/dotnet/sdk]
  5.0.400 [/usr/local/share/dotnet/sdk]
  6.0.100-preview.4.21255.9 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-preview.4.21253.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.4.21253.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

[vzarytovskii]

It seems to be related to single-file/self-contained publish.
Gonna check it.

[baronfel]

Check out this code and comment. This is from the member that's being called that crashes in your repro. It sounds like if you're publishing self-contained then you'll need to set this environment variable to the path of the F# Compiler binaries that you want to use.

[MortalFlesh]

Thanks for quick replies (even on friday!)

as @baronfel mentioned

But at a first glance, I'd say your issue is because I'm the successful scenarios, you're running on the globally installed dotnet SDK, which the FCS component has several assumptions regarding. These assumptions allow it to prove for SDK reference directories. When you publish a self contained app those assumptions are violated, so a workaround would be to investigate the code of that component and ship SDK dlls with your self contained app that mimic the already coded probing logic.

Well first of all my main concern with this issue is that I've done everything the same way a documentation says https://fsharp.github.io/FSharp.Compiler.Service/untypedtree.html and it worked and then stopped without touching a compiled self-contained binary - which somehow smells I guess.

When now I downloaded https://github.com/TypedUseCase/tuc-console/releases/tag/1.5.0 osx-x64.7z and extract it, it has this error and I'm 99% sure it was ok few months ago - that is I don't understand.

And second, I'm not sure how to find those dlls :)

[vzarytovskii]

Check out this code and comment. This is from the member that's being called that crashes in your repro. It sounds like if you're publishing self-contained then you'll need to set this environment variable to the path of the F# Compiler binaries that you want to use.

Yeah, Assembly.GetExecutingAssembly().Location/AppDomain.CurrentDomain.BaseDirectory will likely not work with self-contained apps.

[queil]

I am facing the same issue but using FSHARP_COMPILER_BIN=/usr/share/dotnet/sdk/5.0.400/ I get another exception:

System.NullReferenceException: Object reference not set to an instance of an object.
   at FSharp.Compiler.CompilerConfig.GetTargetFrameworkDirectories@1064.GenerateNext(IEnumerable`1& next)
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System.Collections.IEnumerator.MoveNext()
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at FSharp.Compiler.CompilerConfig.TcConfig.GetTargetFrameworkDirectories()
   at FSharp.Compiler.ErrorLogger.ErrorLoggerExtensions.ReraiseIfWatsonable(Exception exn)
   at FSharp.Compiler.ErrorLogger.ErrorLoggerExtensions.ErrorLogger.ErrorRecovery(ErrorLogger x, Exception exn, Range m)
   at FSharp.Compiler.CompilerConfig.TcConfig.GetTargetFrameworkDirectories()
   at FSharp.Compiler.CompilerImports.TcConfig.MsBuildResolve(TcConfig tcConfig, Tuple`2[] references, ResolveAssemblyReferenceMode mode, Range errorAndWarningRange, Boolean showMessages)
   at FSharp.Compiler.CompilerImports.GetAllDllReferences@572.GenerateNext(IEnumerable`1& next)
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System.Collections.IEnumerator.MoveNext()
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at FSharp.Compiler.CompilerImports.TcAssemblyResolutions.GetAssemblyResolutionInformation(CompilationThreadToken ctok, TcConfig tcConfig)
   at FSharp.Compiler.ScriptClosure.ScriptPreprocessClosure.GetFullClosureOfScriptText(CompilationThreadToken ctok, LegacyReferenceResolver legacyReferenceResolver, String defaultFSharpBinariesDir, String filename, ISourceText sourceText, CodeContext codeContext, Boolean useSimpleResolution, Boolean useFsiAuxLib, Boolean useSdkRefs, FSharpOption`1 sdkDirOverride, LexResourceManager lexResourceManager, FSharpFunc`2 applyCommandLineArgs, Boolean assumeDotNetFramework, FSharpFunc`2 tryGetMetadataSnapshot, ReduceMemoryFlag reduceMemoryUsage, DependencyProvider dependencyProvider)
   at FSharp.Compiler.ScriptClosure.LoadClosure.ComputeClosureOfScriptText(CompilationThreadToken ctok, LegacyReferenceResolver legacyReferenceResolver, String defaultFSharpBinariesDir, String filename, ISourceText sourceText, CodeContext implicitDefines, Boolean useSimpleResolution, Boolean useFsiAuxLib, Boolean useSdkRefs, FSharpOption`1 sdkDir, LexResourceManager lexResourceManager, FSharpFunc`2 applyCompilerOptions, Boolean assumeDotNetFramework, FSharpFunc`2 tryGetMetadataSnapshot, ReduceMemoryFlag reduceMemoryUsage, DependencyProvider dependencyProvider)
   at <StartupCode$FSharp-Compiler-Service>.$Service.e@822-10(BackgroundCompiler _, FSharpOption`1 useSdkRefs, FSharpOption`1 useFsiAuxLib, ISourceText sourceText, FSharpOption`1 sdkDirOverride, FSharpOption`1 previewEnabled, FSharpOption`1 otherFlags, FSharpOption`1 optionsStamp, FSharpOption`1 loadedTimeStamp, String filename, FSharpOption`1 assumeDotNetFramework, CompilationThreadToken ctok, ErrorScope _arg36)
   at <StartupCode$FSharp-Compiler-Service>.$Service.f@826-139(BackgroundCompiler _, FSharpOption`1 useSdkRefs, FSharpOption`1 useFsiAuxLib, ISourceText sourceText, FSharpOption`1 sdkDirOverride, FSharpOption`1 previewEnabled, FSharpOption`1 otherFlags, FSharpOption`1 optionsStamp, FSharpOption`1 loadedTimeStamp, String filename, FSharpOption`1 assumeDotNetFramework, CompilationThreadToken ctok, Unit unitVar)
   at <StartupCode$FSharp-Compiler-Service>.$Service.GetProjectOptionsFromScript@871-1.Invoke(CancellationToken ct)
   at <StartupCode$FSharp-Compiler-Service>.$Reactor.EnqueueAndAwaitOpAsync@220-2.Invoke(CompilationThreadToken ctok)

[MortalFlesh]

I'm not sure if I got it right, but I've tried a simple script

#!/usr/bin/env bash

FSHARP_COMPILER_BIN=$(pwd)/bin/Release/net5.0/FSharp.Core.dll
export FSHARP_COMPILER_BIN

ls -l "$FSHARP_COMPILER_BIN"

dist/osx-x64/BugExample

The ls -l line is there just to check whether the file is really there.

This is a result - still not working.

./runPublished.sh                                                                                                                                          ✔  5.0.400  
-rw-r--r--  1 chromecp  LMC\Domain Users  2929528  8 Jun 22:51 /Users/chromecp/fsharp/fscs-bug-example/bin/Release/net5.0/FSharp.Core.dll
====================
FSCS.BugExample

GetProjectOptionsFromScript.Error: System.NullReferenceException: Object reference not set to an instance of an object.
   at FSharp.Compiler.CompilerConfig.GetTargetFrameworkDirectories@1064.GenerateNext(IEnumerable`1& next)
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System.Collections.IEnumerator.MoveNext()
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at FSharp.Compiler.CompilerConfig.TcConfig.GetTargetFrameworkDirectories()
   at FSharp.Compiler.ErrorLogger.ErrorLoggerExtensions.ReraiseIfWatsonable(Exception exn)
   at FSharp.Compiler.ErrorLogger.ErrorLoggerExtensions.ErrorLogger.ErrorRecovery(ErrorLogger x, Exception exn, Range m)
   at FSharp.Compiler.CompilerConfig.TcConfig.GetTargetFrameworkDirectories()
   at FSharp.Compiler.CompilerImports.TcConfig.MsBuildResolve(TcConfig tcConfig, Tuple`2[] references, ResolveAssemblyReferenceMode mode, Range errorAndWarningRange, Boolean showMessages)
   at FSharp.Compiler.CompilerImports.GetAllDllReferences@572.GenerateNext(IEnumerable`1& next)
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System.Collections.IEnumerator.MoveNext()
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at FSharp.Compiler.CompilerImports.TcAssemblyResolutions.GetAssemblyResolutionInformation(CompilationThreadToken ctok, TcConfig tcConfig)
   at FSharp.Compiler.ScriptClosure.ScriptPreprocessClosure.GetFullClosureOfScriptText(CompilationThreadToken ctok, LegacyReferenceResolver legacyReferenceResolver, String defaultFSharpBinariesDir, String filename, ISourceText sourceText, CodeContext codeContext, Boolean useSimpleResolution, Boolean useFsiAuxLib, Boolean useSdkRefs, FSharpOption`1 sdkDirOverride, LexResourceManager lexResourceManager, FSharpFunc`2 applyCommandLineArgs, Boolean assumeDotNetFramework, FSharpFunc`2 tryGetMetadataSnapshot, ReduceMemoryFlag reduceMemoryUsage, DependencyProvider dependencyProvider)
   at FSharp.Compiler.ScriptClosure.LoadClosure.ComputeClosureOfScriptText(CompilationThreadToken ctok, LegacyReferenceResolver legacyReferenceResolver, String defaultFSharpBinariesDir, String filename, ISourceText sourceText, CodeContext implicitDefines, Boolean useSimpleResolution, Boolean useFsiAuxLib, Boolean useSdkRefs, FSharpOption`1 sdkDir, LexResourceManager lexResourceManager, FSharpFunc`2 applyCompilerOptions, Boolean assumeDotNetFramework, FSharpFunc`2 tryGetMetadataSnapshot, ReduceMemoryFlag reduceMemoryUsage, DependencyProvider dependencyProvider)
   at <StartupCode$FSharp-Compiler-Service>.$Service.e@822-10(BackgroundCompiler _, FSharpOption`1 useSdkRefs, FSharpOption`1 useFsiAuxLib, ISourceText sourceText, FSharpOption`1 sdkDirOverride, FSharpOption`1 previewEnabled, FSharpOption`1 otherFlags, FSharpOption`1 optionsStamp, FSharpOption`1 loadedTimeStamp, String filename, FSharpOption`1 assumeDotNetFramework, CompilationThreadToken ctok, ErrorScope _arg36)
   at <StartupCode$FSharp-Compiler-Service>.$Service.f@826-139(BackgroundCompiler _, FSharpOption`1 useSdkRefs, FSharpOption`1 useFsiAuxLib, ISourceText sourceText, FSharpOption`1 sdkDirOverride, FSharpOption`1 previewEnabled, FSharpOption`1 otherFlags, FSharpOption`1 optionsStamp, FSharpOption`1 loadedTimeStamp, String filename, FSharpOption`1 assumeDotNetFramework, CompilationThreadToken ctok, Unit unitVar)
   at <StartupCode$FSharp-Compiler-Service>.$Service.GetProjectOptionsFromScript@871-1.Invoke(CancellationToken ct)
   at <StartupCode$FSharp-Compiler-Service>.$Reactor.EnqueueAndAwaitOpAsync@220-2.Invoke(CompilationThreadToken ctok)
--- End of stack trace from previous location ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit()
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInAnotherThread[a](CancellationToken token, FSharpAsync`1 computation, FSharpOption`1 timeout)
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout)
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
   at Program.DomainResolverLibrary.parseAndCheck(FSharpChecker checker, String file, String contents)
====================

❗ But the exception is different now. (same as @queil sent)