Add [AssemblyFixtureProvider] for cross-assembly assembly fixtures by Evangelink · Pull Request #8677 · microsoft/testfx

Evangelink · 2026-05-29T07:26:57Z

Closes #757.

Phase 1: opt-in attribute + discovery (no source generator)

Adds a new [AssemblyFixtureProvider(typeof(MyFixture))] assembly-level attribute. A test
assembly that pulls in a library decorated with this marker automatically picks up the
[AssemblyInitialize]/[AssemblyCleanup] methods declared on the referenced fixture type
— without the consumer having to duplicate the method in their own test project.

Why opt-in and library-side?

Pinpointed by the marker rather than crawling every referenced assembly's types, so the
discovery cost is bounded:

Cost	Bound
Walk references	O(direct references), already enumerated by the CLR.
Per assembly: one `GetCustomAttributes` call	O(1) — assembly attrs are tiny.
Per marker: `GetDeclaredMethods(FixtureType)`	O(methods on one named type), no all-types crawl.
Forced extra loads	Avoided — we resolve through the same `AssemblyLoadContext` as the test asm.

Behavior

Local declarations always win. A local [AssemblyInitialize]/[AssemblyCleanup]
snapshots before the provider pass; the provider can only fill empty slots and is
silently ignored if it would overwrite a local fixture.
Cross-provider duplicates throw the existing UTA013/UTA014 diagnostics from
TestAssemblyInfo setters — no behavior change for the conflict diagnostic.
Generic fixture types are rejected with the new UTA070 diagnostic.
Reflection failures on the explicit fixture type are surfaced as UTA071
TypeInspectionException — opt-in markers shouldn't silently disappear.
Marker on the test assembly itself works too, as a consumer-side escape hatch when
the library author cannot ship the attribute.

Discovery details

BFS over Assembly.GetReferencedAssemblies() from the test assembly. We do not iterate
AppDomain.CurrentDomain.GetAssemblies() (which misses passively-referenced libs).
On .NET Core we resolve via AssemblyLoadContext.GetLoadContext(referrer).LoadFromAssemblyName
so plugin-style hosts don't get a second copy of the provider library in the wrong ALC.
BFS visited set keyed on AssemblyName.FullName so multi-version / multi-token
references with the same simple name are not collapsed.
TryLoadReferencedAssembly narrowly catches FileNotFoundException /
FileLoadException / BadImageFormatException — other exceptions propagate.
Framework / MSTest assembly names are skipped by prefix as a fast bail-out; users with
framework-prefixed names can fall back to placing the marker on the test assembly itself.

Tests

Discovers init / cleanup / both from a provider.
Local fixture wins over provider silently; provider fills the other slot if local has
only one.
Wrong signature is still rejected with TypeInspectionException.
Cross-provider duplicate init throws UTA013; duplicate cleanup throws UTA014.
Provider with no fixture methods is a no-op.

All tests pass on net462 / net48 / net8.0 / net9.0 / net8.0-windows.

Out of scope (follow-ups)

Source-generator that auto-emits the marker (Phase 2 in the design doc).
Pre-registered MethodInfo pointers via module initializer (Phase 3).

) Introduces a new opt-in, assembly-level attribute that lets a library expose its `[AssemblyInitialize]` / `[AssemblyCleanup]` methods to every test assembly that references it, without forcing every consumer test project to add a local shim. This is Phase 1 of the design — attribute + discovery only, no source generator yet. New API ------- * `Microsoft.VisualStudio.TestTools.UnitTesting.AssemblyFixtureProviderAttribute` applied at assembly level, pointing at a type that hosts `public static` `[AssemblyInitialize]` / `[AssemblyCleanup]` methods. `AllowMultiple = true`. Discovery --------- `TypeCache.CreateTestAssemblyInfo` now, after the existing in-assembly pass, walks `AssemblyFixtureProviderAttribute` markers on the test assembly itself (consumer escape hatch) and on every other loaded non-framework assembly, then enumerates the marker's `FixtureType` for matching methods. Local declarations in the test assembly always win silently — the provider pass only fills slots that are still empty. Cross-provider duplicates still trigger the existing `UTA_ErrorMultiAssemblyInit` / `UTA_ErrorMultiAssemblyClean` diagnostics. The walk is bounded: * per assembly: a single `GetCustomAttributes` call (O(1) on attribute count), * per marker: `GetDeclaredMethods` on the single named `FixtureType`, * short-circuits as soon as both slots are filled, * framework / MSTest's own assemblies are skipped by name prefix. Tests ----- Six new `TypeCacheAssemblyFixtureProviderTests` cover: * provider supplies init only, * provider supplies cleanup only, * provider supplies both, * local init wins silently over a provider init, * provider fills cleanup when local has only init, * provider method with wrong signature still throws `TypeInspectionException`. Closes part of #757. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Round 1 + round 2 expert review fixes: - Discovery now BFS-walks the test assembly's reference graph instead of AppDomain.CurrentDomain.GetAssemblies(), so provider markers on libraries that haven't been touched yet are also discovered. - Local fixture declarations are snapshotted before the provider pass so they always win silently. Cross-provider duplicates surface as UTA013/UTA014 via the existing setter throws. - Reject generic fixture types (ContainsGenericParameters) with new resource UTA070. - Surface explicit-provider reflection failures as TypeInspectionException (UTA071) instead of silently dropping the fixture. - Use AssemblyLoadContext.GetLoadContext(referrer).LoadFromAssemblyName on .NET Core so plugin-style hosts don't end up with a second copy of the provider library in the wrong ALC. - BFS visited set keyed on AssemblyName.FullName (not simple name) so multi-version/multi-token references aren't collapsed. - Narrow TryLoadReferencedAssembly catch to FileNotFoundException/FileLoadException/BadImageFormatException. - Null-check AssemblyFixtureProviderAttribute(Type) ctor argument. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds an opt-in MSTest assembly fixture provider mechanism for issue #757, allowing referenced libraries to contribute [AssemblyInitialize] and [AssemblyCleanup] methods through [AssemblyFixtureProvider].

Changes:

Adds the public AssemblyFixtureProviderAttribute API and public API tracking.
Extends TypeCache to discover provider markers across referenced assemblies and collect fixture methods.
Adds diagnostics/resources/localization entries and unit tests for provider discovery behavior.

Show a summary per file

File	Description
`src/TestFramework/TestFramework/Attributes/Lifecycle/AssemblyFixtureProviderAttribute.cs`	Defines the new assembly-level provider attribute.
`src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt`	Tracks the new public API surface.
`src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs`	Adds provider discovery and fixture method collection.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/Resource.resx`	Adds UTA070/UTA071 diagnostics.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.cs.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.de.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.es.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.fr.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.it.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.ja.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.ko.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.pl.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.pt-BR.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.ru.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.tr.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.zh-Hans.xlf`	Adds localized resource placeholders.
`src/Adapter/MSTestAdapter.PlatformServices/Resources/xlf/Resource.zh-Hant.xlf`	Adds localized resource placeholders.
`test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TypeCacheAssemblyFixtureProviderTests.cs`	Adds unit coverage for provider discovery and conflict behavior.
`test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Services/AssemblyAttributes.cs`	Registers test assembly provider markers used by the new tests.

Copilot's findings

Files reviewed: 19/19 changed files
Comments generated: 3

Evangelink

Expert Review — PR #8677: `[AssemblyFixtureProvider]` for cross-assembly assembly fixtures

#	Dimension	Verdict
1	Algorithmic Correctness	✅ LGTM
2	Threading & Concurrency	✅ LGTM
3	Security & IPC	✅ LGTM
4	Public API & Binary Compatibility	✅ LGTM
5	Performance & Allocations	✅ LGTM
6	Cross-TFM Compatibility	✅ LGTM
7	Resource & IDisposable Management	✅ LGTM
8	Defensive Coding at Boundaries	✅ LGTM
9	Localization & Resources	✅ LGTM
10	Test Isolation	✅ LGTM
11	Assertion Quality	✅ LGTM
12	Flakiness Patterns	✅ LGTM
13	Test Completeness & Coverage	🟠 MODERATE
14	Data-Driven Test Coverage	✅ N/A
15	Code Structure & Simplification	✅ LGTM
16	Naming & Conventions	✅ LGTM
17	Documentation Accuracy	✅ LGTM
18	Analyzer & Code Fix Quality	✅ N/A
19	IPC Wire Compatibility	✅ N/A
20	Build Infrastructure & Dependencies	✅ LGTM
21	Scope & PR Discipline	✅ LGTM

✅ 20/21 dimensions clean.

Test Completeness (MODERATE) — Two new diagnostic codes (UTA070, UTA071) and two logic branches have no test coverage. See inline comment for details.

Generated by Expert Code Review (on open) for issue #8677 · sonnet46 5.7M

Comments that could not be inline-anchored

test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TypeCacheAssemblyFixtureProviderTests.cs:370

Test Completeness — MODERATE

Four code paths introduced in this PR have no test coverage:

UTA070 — Generic fixture type (TypeCache.cs lines 490–494, fixtureType.ContainsGenericParameters → throw):

public void ShouldThrowUTA070WhenFixtureTypeIsGeneric()
{
    // arrange: mock GetCustomAttributes to return AssemblyFixtureProvider(typeof(GenericFixture&lt;&gt;))
    // act + assert: Throw&lt;TypeInspectionException&gt;().WithMessage(&quot;*UTA070*&quot;)
}

**UTA07…

- Fix UWP/AssemblyLoadContext guards (NET && !WINDOWS_UWP) - Replace swallow-all catch with CustomAttributeData presence check + UTA072 diagnostic for AssemblyFixtureProvider load failures - Replace generic catch in SafeGetAssemblyName with filtered catch - Add UTA_AssemblyFixtureProviderLoadFailed resource (resx + xlf) - Add acceptance test covering cross-assembly fixture provider flow Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ovider

Copilot

Copilot's findings

Files reviewed: 20/20 changed files
Comments generated: 2

Pass --settings my.runsettings to ExecuteAsync so CaptureTraceOutput=false forwards Console.WriteLine output to stdout, allowing the assertions for the assembly init/cleanup messages to match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Reject closed generic provider types (e.g. typeof(Foo<int>)) in addition to open generics. Use Type.IsGenericType which covers both cases. Addresses copilot review comment 3330711195. - Extract LoadProviderMarkers, ProcessProviderFixtureType, and elevate CollectFixtureMethodsFromProviderType to internal static so the UTA070/ UTA071/UTA072 paths can be exercised by direct unit tests. - Add four new TypeCache tests covering both open and closed generic rejection, reflection failure during method enumeration, and attribute instantiation failure. Addresses Expert Review test-completeness finding (MODERATE). - Clarify the EnumerateCandidateAssemblies comment: GetReferencedAssemblies walks the metadata reference table, not arbitrary project-file references. Addresses copilot review comment 3330711185. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Evangelink · 2026-06-01T08:12:53Z

Addressed the Test Completeness (MODERATE) finding from the expert review in 2e30ce9. Added four new unit tests in TypeCacheAssemblyFixtureProviderTests covering the previously untested diagnostics:

ProcessProviderFixtureTypeShouldThrowUTA070ForOpenGenericFixtureType – typeof(MyFixture<>) rejected.
ProcessProviderFixtureTypeShouldThrowUTA070ForClosedGenericFixtureType – typeof(MyFixture<int>) rejected (this case was actually a real gap – ContainsGenericParameters returned false for closed generics; the check now uses IsGenericType).
CollectFixtureMethodsFromProviderTypeShouldThrowUTA071WhenMethodEnumerationFails – simulates GetDeclaredMethods failing and asserts UTA071 surfaces.
LoadProviderMarkersShouldThrowUTA072WhenAttributeInstantiationFails – simulates the attribute instantiation throwing (e.g. typeof(...) argument unresolvable) and asserts UTA072 surfaces with the original exception as the inner exception.

To make these reachable from unit tests without polluting AssemblyAttributes.cs with broken markers, the inner validation/loading helpers (ProcessProviderFixtureType, CollectFixtureMethodsFromProviderType, LoadProviderMarkers) were elevated to internal static so tests can call them directly.

All 851 tests in MSTestAdapter.PlatformServices.UnitTests pass.

Copilot

Copilot's findings

Files reviewed: 20/20 changed files
Comments generated: 1

+
+            object[] markers = LoadProviderMarkers(candidate);
+
+            if (markers is null || markers.Length == 0)


Evangelink and others added 2 commits May 28, 2026 19:53

Copilot AI review requested due to automatic review settings May 29, 2026 07:26

Copilot started reviewing on behalf of Evangelink May 29, 2026 07:27 View session

microsoft-github-policy-service Bot added the area/fixtures Test class / assembly initialization & cleanup. label May 29, 2026

Copilot AI reviewed May 29, 2026

View reviewed changes

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs Outdated

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs Outdated

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs

github-code-quality Bot found potential problems May 29, 2026

View reviewed changes

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs Fixed

Evangelink commented May 29, 2026

View reviewed changes

Amaury Levé and others added 2 commits May 31, 2026 08:58

Merge branch 'main' into dev/amauryleve/issue-757-assembly-fixture-pr…

59d5ba6

…ovider

Copilot AI review requested due to automatic review settings May 31, 2026 18:14

Copilot started reviewing on behalf of Evangelink May 31, 2026 18:14 View session

Copilot AI reviewed May 31, 2026

View reviewed changes

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs Outdated

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs Outdated

Evangelink and others added 2 commits May 31, 2026 20:26

Copilot AI review requested due to automatic review settings June 1, 2026 08:12

Copilot started reviewing on behalf of Evangelink June 1, 2026 08:12 View session

Copilot AI reviewed Jun 1, 2026

View reviewed changes

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs

object[] markers = LoadProviderMarkers(candidate);

if (markers is null || markers.Length == 0)

Evangelink merged commit 0196096 into main Jun 1, 2026
43 of 47 checks passed

Evangelink deleted the dev/amauryleve/issue-757-assembly-fixture-provider branch June 1, 2026 08:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add [AssemblyFixtureProvider] for cross-assembly assembly fixtures#8677

Add [AssemblyFixtureProvider] for cross-assembly assembly fixtures#8677
Evangelink merged 6 commits into
mainfrom
dev/amauryleve/issue-757-assembly-fixture-provider

Evangelink commented May 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Evangelink left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Evangelink commented Jun 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		object[] markers = LoadProviderMarkers(candidate);

		if (markers is null \|\| markers.Length == 0)

Conversation

Evangelink commented May 29, 2026

Phase 1: opt-in attribute + discovery (no source generator)

Why opt-in and library-side?

Behavior

Discovery details

Tests

Out of scope (follow-ups)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Evangelink left a comment

Choose a reason for hiding this comment

Expert Review — PR #8677: [AssemblyFixtureProvider] for cross-assembly assembly fixtures

Comments that could not be inline-anchored

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Evangelink commented Jun 1, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Expert Review — PR #8677: `[AssemblyFixtureProvider]` for cross-assembly assembly fixtures