Add Assert.ContainsAll and Assert.DoesNotContainAll with structured assertion messages

[Evangelink]

Summary

Adds Assert.ContainsAll and Assert.DoesNotContainAll (generic and non-generic, with and without an IEqualityComparer) on the Assert class, providing the same containment semantics as CollectionAssert.IsSubsetOf / CollectionAssert.IsNotSubsetOf but with the RFC 012 structured assertion message format and naming consistent with the existing Assert.Contains(expected, collection) API.

This is the first in a small series of PRs lifting CollectionAssert.* APIs onto Assert.* with the new message format.

API surface

8 new public overloads:

// Generic
Assert.ContainsAll<T>(IEnumerable<T>? expected, IEnumerable<T>? collection, string? message = "", ...)
Assert.ContainsAll<T>(IEnumerable<T>? expected, IEnumerable<T>? collection, IEqualityComparer<T>? comparer, string? message = "", ...)
Assert.DoesNotContainAll<T>(IEnumerable<T>? expected, IEnumerable<T>? collection, string? message = "", ...)
Assert.DoesNotContainAll<T>(IEnumerable<T>? expected, IEnumerable<T>? collection, IEqualityComparer<T>? comparer, string? message = "", ...)

// Non-generic
Assert.ContainsAll(IEnumerable? expected, IEnumerable? collection, string? message = "", ...)
Assert.ContainsAll(IEnumerable? expected, IEnumerable? collection, IEqualityComparer? comparer, string? message = "", ...)
Assert.DoesNotContainAll(IEnumerable? expected, IEnumerable? collection, string? message = "", ...)
Assert.DoesNotContainAll(IEnumerable? expected, IEnumerable? collection, IEqualityComparer? comparer, string? message = "", ...)

All overloads support [CallerArgumentExpression] capture for expected and collection.

The parameter naming mirrors the existing Assert.Contains(expected, collection) API: expected is the (collection of) item(s) we expect to find, collection is the container being searched.

Behavior

• Multiplicity-aware diffing: each excess element in expected (with multiplicity) appears in the missing: evidence line — a true diff rather than a set-membership check.
• Null handling: nulls in expected are matched against nulls in collection (by count) without going through the user-supplied IEqualityComparer, matching the existing CollectionAssert semantics.
• Single enumeration: expected and collection are snapshotted to List<T?> once before counting, so lazy / single-pass enumerables behave deterministically and the failure path doesn't re-enumerate.
• Comparer in evidence + call-site: when a non-default comparer is supplied, RFC 012 requires the comparer's short type name in the evidence block and a <comparer> placeholder in the rendered call-site. Both are emitted.

Example failure messages

Assertion failed. Expected collection to contain all specified items.

missing:    [3]
expected:   [1, 2, 3]
collection: [1, 2]

Assert.ContainsAll(new[] { 1, 2, 3 }, new[] { 1, 2 })

With a non-default comparer:

Assertion failed. Expected collection to contain all specified items.

missing:    ["C"]
expected:   ["A", "C"]
collection: ["a", "b"]
comparer:   OrdinalIgnoreCaseComparer

Assert.ContainsAll(new[] { "A", "C" }, new[] { "a", "b" }, <comparer>)

Tests

35+ unit tests in AssertTests.ContainsAll.cs covering:

• Passing scenarios (all present, empty expected, multiplicity within collection).
• Failing scenarios with full structured-message verification.
• Multiplicity exceeding collection.
• Nulls in expected, nulls only in collection.
• Custom IEqualityComparer<T> (StringComparer.OrdinalIgnoreCase) + comparer-line + <comparer> placeholder in call-site.
• Non-generic IEnumerable variants (with and without comparer).
• All [NotNull] parameter contracts (expected, collection, comparer) for all 8 overloads.
• Empty-collection edge cases for DoesNotContainAll (both empty / empty expected both correctly fail).
• AssertFailedException.ExpectedText / ActualText properties are populated correctly.

All tests pass; full repo Build.cmd succeeds with 0 warnings, 0 errors.

Notes

• 2 new resource entries in FrameworkMessages.resx; .xlf files were regenerated by the build (not hand-edited).
• 8 new entries added to PublicAPI.Unshipped.txt.
• No existing public API was modified.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds new Assert.IsSubsetOf / Assert.IsNotSubsetOf overloads (generic + non-generic, with optional comparers) using RFC 012 structured assertion messages, plus tests and localized resource strings.

Changes:

• Introduces 8 new Assert.*SubsetOf* overloads with structured evidence/call-site rendering.
• Adds a dedicated unit test suite validating pass/fail behavior, null handling, comparer rendering, and AssertFailedException Expected/Actual text population.
• Adds new message resources (IsSubsetOfFailedSummary, IsNotSubsetOfFailedSummary) and updates localization files + public API declarations.

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.IsSubsetOf.cs	Adds coverage for new subset/not-subset asserts including structured message verification.
src/TestFramework/TestFramework/Assertions/Assert.IsSubsetOf.cs	Implements the new Assert APIs, diffing logic, evidence creation, and comparer adaptation for non-generic enumerables.
src/TestFramework/TestFramework/Resources/FrameworkMessages.resx	Adds 2 new message summary resources used by structured assertion messages.
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.*.xlf	Propagates the 2 new resource entries into localized XLF files.
src/TestFramework/TestFramework/PublicAPI/PublicAPI.Unshipped.txt	Declares the 8 new public overloads for API tracking.

Copilot's findings

• Files reviewed: 17/17 changed files
• Comments generated: 4

[Evangelink]

PR #8234 — Assert.IsSubsetOf / Assert.IsNotSubsetOf

Good implementation. Multiplicity-aware diffing, single-enumeration snapshots, null-safe counting, proper structured messages, correct PublicAPI.Unshipped.txt entries, auto-generated XLF. The test coverage is solid (35 tests covering generic/non-generic, comparer, null inputs, edge cases, and ExpectedText/ActualText population).

---

Verdict Table

#	Dimension	Status	Notes
1	Algorithmic Correctness	✅ LGTM	Empty subset, null handling, multiplicity all correct
2	Threading & Concurrency	✅ N/A	No shared mutable state
3	Security	✅ N/A	No IPC/deserialization
4	Public API & Binary Compat	✅ LGTM	8 entries in PublicAPI.Unshipped.txt, no init accessors, sealed class
5	Performance & Allocations	✅ LGTM	Single enumeration, List<T?> snapshot reuse
6	Cross-TFM Compatibility	✅ LGTM	No TFM-gated API used
7	Resource & IDisposable	✅ N/A	No disposables
8	Defensive Coding	✅ LGTM	All params null-checked before use
9	Localization & Resources	✅ LGTM	.resx updated, XLF auto-generated
10	Test Isolation	✅ LGTM	No shared static mutable state
11	Assertion Quality	✅ LGTM	Uses TestFramework.ForTestingMSTest + AwesomeAssertions
12	Flakiness	✅ N/A	No sleeps or order-dependent collection asserts
13	Test Completeness	✅ LGTM	All 8 overloads exercised; null/edge cases covered
14	Data-Driven Coverage	✅ N/A	Direct tests sufficient
15	Code Structure	✅ LGTM	Pattern matching, is null/is not null throughout
16	Naming & Conventions	✅ LGTM	File-scoped namespace, consistent with codebase
17	Documentation	✅ LGTM	Full XML docs on all 8 public overloads
18	Analyzer Quality	✅ N/A	No analyzer changes
19	IPC Wire Compat	✅ N/A	No serialization
20	Build Infrastructure	✅ N/A	No eng/ changes
21	Scope & PR Discipline	✅ LGTM	Single focused concern

---

Key Findings

[MODERATE] BuildCallSiteWithComparer (line 454) performs string surgery by stripping the trailing ) from FormatCallSiteExpression's output and appending , <comparer>). This silently produces a malformed call-site string if the format ever changes. The best fix is a new FormatCallSiteExpression overload that accepts a third argument expression; a defensive Debug.Assert(callSite.EndsWith(')')) is a reasonable short-term guard. See inline comment.

[NIT] public new bool Equals in NonGenericEqualityComparerAdapter is correct and intentional (implements IEqualityComparer<object?>), but the new keyword warrants a brief comment so future readers don't question it. See inline comment.

[NIT — test gap] The non-generic + comparer happy-path cases (IsSubsetOf_NonGeneric_WithComparer_AllPresent_ShouldPass and IsNotSubsetOf_NonGeneric_WithComparer_MissingElement_ShouldPass) are absent. Not blocking, but the symmetry is worth closing.

Generated by Expert Code Review (on open) for issue #8234 · ● 10.7M

[Copilot]

Copilot's findings

• Files reviewed: 18/18 changed files
• Comments generated: 1

[Copilot]

Copilot's findings

• Files reviewed: 18/18 changed files
• Comments generated: 1

[Evangelink]

Good catch. Fixed in d007f56 by passing "<comparer>" (instead of string.Empty) as expression3 in BuildCallSiteWithComparer. Because the comparer parameter doesn't carry a [CallerArgumentExpression], the third slot was always empty, which combined with the all-empty short-circuit in FormatCallSiteExpression could suppress the call site entirely when callers (e.g. wrappers or non-C# languages) don't propagate caller expressions. With this change the call site now always renders, and degrades to Assert.ContainsAll(<expected>, <collection>, <comparer>) (and likewise for DoesNotContainAll) when no expressions are available. Added two unit tests in AssertTests.ContainsAll.cs covering the missing-caller-info case for both APIs.