Fix flaky type-provider test in MultiProjectAnalysisTests by isolating its checker and disabling parallelization

[Copilot]

The test In-memory cross-project references to projects using generative type provides should fallback to on-disk references shared the module-level checker instance with all other tests in MultiProjectAnalysisTests.fs, causing intermittent CI failures due to parallel execution races when type providers are involved — particularly on net472 where type providers run in-process.

Changes

• MultiProjectAnalysisTests.fs:
  • Wrapped the test in a [<Collection(nameof NotThreadSafeResourceCollection)>] module (GenerativeTypeProviderFallbackTest), which enforces DisableParallelization = true via xUnit's collection mechanism. This prevents the test from running in parallel with other test collections, avoiding type provider assembly loading races on net472. This follows the same pattern used by ClearLanguageServiceRootCachesTest and ScriptClosureCacheUse in ProjectAnalysisTests.fs.
  • Added a dedicated FSharpChecker instance scoped to the test function, shadowing the shared module-level checker for all ParseAndCheckFileInProject calls within the test.

[<Collection(nameof NotThreadSafeResourceCollection)>]
module GenerativeTypeProviderFallbackTest =

    [<Fact>]
    let ``In-memory cross-project references to projects using generative type provides should fallback to on-disk references`` () =
        ...
        // Dedicated checker to isolate type-provider tests from shared state races.
        let checker = FSharpChecker.Create(useTransparentCompiler = CompilerAssertHelpers.UseTransparentCompiler)
        ...

Original prompt

Problem

Following the same pattern as PR #19393 which isolated the checker for Project25 type provider tests in ProjectAnalysisTests.fs, we need to apply the same fix to tests/FSharp.Compiler.Service.Tests/MultiProjectAnalysisTests.fs.

The test In-memory cross-project references to projects using generative type provides should fallback to on-disk references (starting at line 818) uses a type provider (TestTP) and currently uses the shared module-level checker (defined at line 20). This can cause flaky failures in CI due to shared state races with other tests running in parallel that also use the same checker instance.

Required Changes

In the file tests/FSharp.Compiler.Service.Tests/MultiProjectAnalysisTests.fs:

1. Add [<RunTestCasesInSequence>] attribute to the test In-memory cross-project references to projects using generative type provides should fallback to on-disk references (line 818). Change [<Fact>] to [<Fact; RunTestCasesInSequence>].

2. Create a dedicated checker instance for this test. Since the test is a standalone function (not in a module like Project25), add a dedicated checker inside the test function itself, right after the tpDLL/csDLL setup and File.Exists checks. Then replace all uses of the shared checker in that test with the local dedicated one.

Here is the exact pattern from PR #19393 that was applied to ProjectAnalysisTests.fs:

• A dedicated checker was added: let checker = FSharpChecker.Create(useTransparentCompiler = FSharp.Test.CompilerAssertHelpers.UseTransparentCompiler)
• All checker. calls within the type-provider tests were changed to use this dedicated checker instead of the shared module-level one.

Apply the same approach to the single type-provider test in MultiProjectAnalysisTests.fs:

In the test function In-memory cross-project references to projects using generative type provides should fallback to on-disk references:

• Add RunTestCasesInSequence to the attribute: change [<Fact>] to [<Fact; RunTestCasesInSequence>]
• After the tpDLL and csDLL existence checks (after line 828), add:

  // Dedicated checker to isolate type-provider tests from shared state races.
  let checker = FSharpChecker.Create(useTransparentCompiler = FSharp.Test.CompilerAssertHelpers.UseTransparentCompiler)

  This will shadow the module-level checker, so all subsequent uses of checker within the test function body will use the dedicated instance instead of the shared one. The three places where checker is used in this test (lines 907, 927, 952 — all checker.ParseAndCheckFileInProject(...) calls) will automatically pick up the local binding.

No other tests in this file need changes — only this one test uses type providers.

This pull request was created from Copilot chat.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[github-actions]

✅ No release notes required

[T-Gro]

@copilot :

failed FSharp.Compiler.Service.Tests.MultiProjectAnalysisTests.In-memory cross-project references to projects using generative type provides should fallback to on-disk references (25s 858ms)
Assert.Equal() Failure: Values differ
Expected: False
Actual: True
from D:\a_work\1\s\artifacts\bin\FSharp.Compiler.Service.Tests\Release\net472\FSharp.Compiler.Service.Tests.exe (net48|x64)
Assert.Equal() Failure: Values differ
Expected: False
Actual: True
at FSharp.Compiler.Service.Tests.MultiProjectAnalysisTests.In-memory cross-project references to projects using generative type provides should fallback to on-disk references() in D:\a_work\1\s\tests\FSharp.Compiler.Service.Tests\MultiProjectAnalysisTests.fs:925
Standard output
FSharp.Compiler.Service.Tests.exe Information: 0 : FCS: Unknown.CreateOneIncrementalBuilder (D:\a_work\1\s\tests\FSharp.Compiler.Service.Tests../service/data/TestProject/TestProject.fsproj)
FSharp.Compiler.Service.Tests.exe Information: 0 : FCS: Unknown.parseFile (D:\a_work\1\s\tests\FSharp.Compiler.Service.Tests../service/data/TestProject/TestProject.fs)
Parse Diagnostics (TestProject): [||]
Check Diagnostics (TestProject): [|D:\a_work\1\s\tests\FSharp.Compiler.Service.Tests../service/data/TestProject/TestProject.fs (6,22)-(6,37) typecheck error The type provider 'TestTP.BasicProvider' reported an error in the context of provided type 'ErasedWithConstructor.Provided.MyType', member 'DoNothing'. The error: Method 'Void doNothing()' not found in type ''. This method may be missing in the types available in the target assemblies.|]

Error output

[Copilot]

@copilot :

failed FSharp.Compiler.Service.Tests.MultiProjectAnalysisTests.In-memory cross-project references to projects using generative type provides should fallback to on-disk re...

The RunTestCasesInSequence attribute is just a marker for future xUnit 3 — it doesn't actually prevent parallel execution in xUnit 2. The TP error (Method 'Void doNothing()' not found) is a race condition from concurrent TP assembly loading on net472 (where TPs run in-process).

In 3af1a33, I wrapped the test in a [<Collection(nameof NotThreadSafeResourceCollection)>] module, which enforces DisableParallelization = true via xUnit's collection mechanism — the same pattern used by ClearLanguageServiceRootCachesTest and ScriptClosureCacheUse in ProjectAnalysisTests.fs. This prevents the test from running in parallel with any other collection, avoiding the TP assembly loading race.