Using multiple FsiEvaluationSessions in the same process causes thread-safety issues

[tastyeggs]

While I understand that an FsiEvaluationSession is by itself not thread-safe, there doesn't seem to be any documentation that suggests that running multiple sessions within the same process space should conflict.

However, when I run multiple sessions in parallel, under high stress, some evaluations fail with the following exception:

Message: System.Exception : Error creating evaluation session: System.InvalidOperationException: Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.
   at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
   at System.Collections.Generic.Dictionary`2.TryGetValue(TKey key, TValue& value)
   at FSharp.Compiler.AbstractIL.Internal.Library.Tables.memoize@1051.Invoke(a x)
   at FSharp.Compiler.AbstractIL.ILBinaryReader.typeDefReader@1784.Invoke(Int32 idx)
   at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Func`1 valueFactory)
   at FSharp.Compiler.Import.entities@482-2.Invoke(Tuple`2 tupledArg)
   at FSharp.Compiler.Import.multisetDiscriminateAndMap@425.GenerateNext(IEnumerable`1& next)
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at FSharp.Compiler.Import.multisetDiscriminateAndMap[Key,Value,a](FSharpFunc`2 nodef, FSharpFunc`2 tipf, FSharpList`1 items)
   at FSharp.Compiler.Import.ImportILTypeDefList[a](FSharpFunc`2 amap, range m, CompilationPath cpath, FSharpList`1 enc, FSharpList`1 items)
   at FSharp.Compiler.Import.modty@480.Invoke(Unit unitVar)
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at FSharp.Compiler.CompileOps.TcImports.ccuHasType(CcuThunk ccu, FSharpList`1 nsname, String tname)
   at Microsoft.FSharp.Collections.ArrayModule.loop@739-41[T](FSharpFunc`2 predicate, T[] array, Int32 i)
   at FSharp.Compiler.TcGlobals.TcGlobals.findSysTypeCcu(FSharpList`1 path, String typeName)
   at FSharp.Compiler.TcGlobals.TcGlobals.findSysTyconRef(FSharpList`1 path, String nm)
   at FSharp.Compiler.TcGlobals.TcGlobals..ctor(Boolean compilingFslib, ILGlobals ilg, CcuThunk fslibCcu, String directoryToResolveRelativePaths, Boolean mlCompatibility, Boolean isInteractive, FSharpFunc`2 tryFindSysTypeCcu, Boolean emitDebugInfoInQuotations, Boolean noDebugData)
   at FSharp.Compiler.CompileOps.BuildFrameworkTcImports@4752-2.Invoke(Tuple`2 _arg7)
   at FSharp.Compiler.AbstractIL.Internal.Library.CancellableModule.bind@709.Invoke(CancellationToken ct)
   at FSharp.Compiler.AbstractIL.Internal.Library.CancellableModule.bind@709.Invoke(CancellationToken ct)
   at FSharp.Compiler.AbstractIL.Internal.Library.CancellableModule.bind@709.Invoke(CancellationToken ct)
   at FSharp.Compiler.AbstractIL.Internal.Library.CancellableModule.runWithoutCancellation[a](Cancellable`1 comp)
   at FSharp.Compiler.Interactive.Shell.FsiEvaluationSession..ctor(FsiEvaluationSessionHostConfig fsi, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, Boolean fsiCollectible, FSharpOption`1 legacyReferenceResolver)

Repro steps

1. Create 5 threads, that each create an FsiEvaluationSession:

let fsiConfig = FsiEvaluationSession.GetDefaultConfiguration()
let sbOut = new StringBuilder()
let sbErr = new StringBuilder()
let inStream = new StringReader("")
let outStream = new StringWriter(sbOut)
let errStream = new StringWriter(sbErr)
let fsiSession = FsiEvaluationSession.Create(fsiConfig, allArgs, inStream, outStream, errStream, collectible = true)

2. Evaluate any expression that deals with types (e.g. {A = 5; B = "Foo"}) a few dozen times on each thread
3. Dispose off the session, and go to Prevent unconditional debug output #1, and re-create the session, and keep repeating

Expected behavior

The evaluation should work without issues.

Actual behavior

You might encounter an arbitrary crash, with the above stack trace, usually within the first few hundred evaluations.

Known workarounds

A global lock is needed around all FSI API calls, both creation and evaluation. One thing to note is even thought the stack trace seems to originate from the constructor of an FsiEvaluationSession, due to a lazy being part of the stack trace, the exception could get thrown as part of regular evaluation.

Related information

Provide any related information

• Operating system - Windows
• Branch - master, FSharp.Compiler.Services 27.0.1, FSharp.Core 4.5.4
• .NET Runtime, CoreCLR or Mono Version - .NET Core 2.1
• Editing Tools (e.g. Visual Studio Version) - VS 2017 15.9.4

[goswinr]

I had a similar question.
#798
I think its not recommended.

"(...) it's not really possible to do this stuff safely(...)"

[tastyeggs]

Ah, thanks @goswinr, I clearly need to improve my searching skills :)

I'll close this as a duplicate in this case.