Apply structured assertion messages (RFC 012) to AreEqual / AreNotEqual by Evangelink · Pull Request #8231 · microsoft/testfx

Evangelink · 2026-05-14T17:04:58Z

Summary

Continues the rollout of RFC 012 — Structured Assertion Messages. Following the recently merged migration of IsTrue / IsFalse / IsNull / IsNotNull (#8187), this PR migrates the Assert.AreEqual / Assert.AreNotEqual family to the structured-message format.

Output format

AreEqual (generic / object):

Assertion failed. Expected values to be equal.

expected: 1
actual:   2

Assert.AreEqual(1, 2)

AreEqual (different types — type info now on its own evidence line):

Assertion failed. Expected values to be equal, but they are of different types.

expected:      System.Object
expected type: System.Object
actual:        1
actual type:   System.Int32

Assert.AreEqual(new object(), 1)

AreEqual (string — keeps the existing length/diff-index summary line):

Assertion failed. Expected strings to be equal.
String lengths are both 201 but differ at index 100.

expected: "..."
actual:   "..."

Assert.AreEqual(expected, actual)

AreNotEqual:

Assertion failed. Expected values to not be equal.

notExpected: 0
actual:      0

Assert.AreNotEqual(0, 0)

AssertFailedException.ExpectedText / ActualText and Exception.Data["assert.expected"] / ["assert.actual"] are populated. For AreNotEqual, ExpectedText is prefixed with not (e.g. "not 0") so the property accurately conveys the failed expectation.

Notable design points

New FormatBinaryCallSiteExpression(method, expr1, paramName1, expr2, paramName2) helper on Assert. When an argument expression contains line breaks (multi-line raw-string literal), it is replaced with a <paramName> placeholder so the call-site line stays single-line: Assert.AreEqual(<expected>, "different").
EvidenceBlock builder padding handles the variable-width labels uniformly across expected: / actual: / expected type: / actual type: / notExpected:.
New resx keys: AreEqualFailedSummary, AreEqualDifferentTypesFailedSummary, AreEqualStringsFailedSummary, AreNotEqualFailedSummary. .xlf files were regenerated via UpdateXlf; no manual edits.
RFC: dropped the misleading (case-sensitive) suffix from string-equal summaries (5 occurrences in docs/RFCs/012-Structured-Assertion-Messages.md) since the same format applies to ordinal-ignore-case overloads as well.

Tests

test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreEqualTests.cs:

All existing AreEqual / AreNotEqual failure tests rewritten to assert the full structured message with .Should().Be("""…""") (raw strings, no Environment.NewLine, no .Contain).
5 new regression tests: AreNotEqual_PopulatesExpectedAndActualTextWithNotPrefix, AreNotEqual_FailsWithStructuredMessage, AreEqual_MultilineExpectedExpression_UsesPlaceholderInCallSite, AreEqual_MultilineActualExpression_UsesPlaceholderInCallSite, AreNotEqual_MultilineNotExpectedExpression_UsesPlaceholderInCallSite.
Long-string test renamed AreEqualLongStringsShouldTruncateAndShowContext → AreEqualLongStringsShowsFullStrings to reflect actual behavior (truncation/preview is deferred to a follow-up PR).

test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs:

All AreEqual-based scope soft-failure tests updated to the new structured message and switched from .Contain(...) to full .Be("""…""") equality.

Build.cmd clean (0 warnings, 0 errors); TestFramework.UnitTests 870/870 passing on net9.0.

Out of scope (deferred to follow-up PRs)

Typed-string overload Assert.AreEqual(string?, string?, bool ignoreCase, ...) still uses the legacy format.
Numeric-delta AreNotEqual interpolated-string handlers (float / double / decimal / long with delta) still call the legacy BuildUserMessageForNotExpectedExpressionAndActualExpression path.
Long-string truncation / preview window (StringPreviewHelper-style caret rendering).

Reviewer iterations

Three rounds of expert-reviewer feedback have been applied (commits 62c39f5fa, 1b7bc4a3f, ae0742c9d, 46a001652):

H1 — AreNotEqual was setting equal ExpectedText/ActualText; now prefixed with not .
M2 — Multi-line raw-string expressions now substitute a <paramName> placeholder instead of dropping the entire call-site line.
L3 — Type info split onto its own evidence line for the different-types case (better alignment / cleaner read).
N1/N2 — IsNullOrEmpty (vs IsNullOrWhiteSpace); Type.FullName is treated as nullable with ?? type.Name fallback.
L1/L2 — Added regression tests covering ExpectedText/ActualText propagation and full-message rendering for AreNotEqual + multi-line placeholder substitution.
N4/N5/N6 — Cached GetType(), hoisted line-break char array to a static readonly, consolidated the duplicated builder block in the string branch.

Introduce the foundational types and helpers for structured multi-line assertion failure messages as described in RFC 012: - EvidenceLine: labeled line record struct for evidence blocks - EvidenceBlock: collection of labeled lines with automatic alignment - StructuredAssertionMessage: builder producing the new multi-line format (prefix + summary + user message + evidence block + call-site) - AssertionValueRenderer: renders values per RFC 012 rules (null, quoted strings with escape sequences, booleans, collections as JSON arrays) - AssertFailedException: add ExpectedText/ActualText public properties - Assert: add ReportAssertFailed/ThrowAssertFailed overloads accepting StructuredAssertionMessage No existing assertion methods are changed yet - this PR only introduces the infrastructure that subsequent PRs will use to migrate each assertion method to the new format.

…Null - Update Assert.IsTrue/IsFalse to use StructuredAssertionMessage with evidence block showing actual value and call-site expression - Update Assert.IsNull to use StructuredAssertionMessage with evidence block showing actual value - Update Assert.IsNotNull to use StructuredAssertionMessage without evidence block (actual is always null per RFC) - Update interpolated string handlers to store condition/value for passing to the new reporting methods - Add FormatCallSiteExpression helper to Assert.cs for formatting call-site display lines - Remove unused BuildUserMessageForConditionExpression method - Update all related test expectations to match new message format

…/AreNotEqual) - Update generic AreEqual<T>/AreNotEqual<T> to use StructuredAssertionMessage - Update string AreNotEqual to use structured format - Update interpolated string handlers for generic equality assertions - Preserve expression text in numeric non-generic handler closures - Update all related test expectations to match new format

…ured-messages-equality # Conflicts: # src/TestFramework/TestFramework/Assertions/Assert.cs # src/TestFramework/TestFramework/Assertions/AssertionValueRenderer.cs # src/TestFramework/TestFramework/Assertions/EvidenceBlock.cs # src/TestFramework/TestFramework/Assertions/StructuredAssertionMessage.cs # test/UnitTests/TestFramework.UnitTests/Assertions/AssertFailedExceptionTests.cs # test/UnitTests/TestFramework.UnitTests/Assertions/AssertionValueRendererTests.cs # test/UnitTests/TestFramework.UnitTests/Assertions/EvidenceBlockTests.cs # test/UnitTests/TestFramework.UnitTests/Assertions/StructuredAssertionMessageTests.cs

…ality

Replace hardcoded English summary literals in IsTrue/IsFalse/IsNull/IsNotNull/AreEqual/AreNotEqual with FrameworkMessages resource lookups. Add new resx keys for AreEqual/AreNotEqual variants. Add WithExpectedAndActual to AreNotEqual structured message. Restore unrelated files that were silently corrupted by merge auto-resolution: Platform ServiceProvider.ReplaceService and ServerTelemetry registration, .github workflow lock files and copilot-instructions, and SoftAssertionTests.

…, equalize DoesNotReturn, restore notExpected evidence - Drop '(case-sensitive)' from AreEqualStringsFailedSummary since the generic AreEqual<T> path also accepts case-insensitive comparers. - Add [DoesNotReturn] to ReportAssertIsTrueFailed and ReportAssertIsNullFailed for symmetry with their IsFalse/IsNotNull siblings. - Restore notExpected: line in AreNotEqual evidence so failures with custom comparers retain diagnostic info.

…ured-messages-equality # Conflicts: # src/TestFramework/TestFramework/Assertions/Assert.IsNull.cs # src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs # src/TestFramework/TestFramework/Resources/FrameworkMessages.resx # test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.IsNull.cs # test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.IsTrueTests.cs # test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs

…pe.FullName, L1+L3 regression tests)

…al full-message test, N4 cache GetType, N5 hoist line break chars, N6 consolidate string branch)

Copilot

Pull request overview

Rolls RFC 012 structured assertion messages forward by migrating Assert.AreEqual / Assert.AreNotEqual failures to the structured-message format (summary + optional user message + aligned evidence block + call-site expression), updating tests and localized resources accordingly.

Changes:

Updated AreEqual / AreNotEqual failure reporting to use StructuredAssertionMessage, including evidence blocks and call-site formatting (with multiline-expression placeholders).
Rewrote/added unit tests to assert full structured messages and expected/actual metadata propagation.
Added new localized resource keys for the new summary lines and updated the RFC text examples.

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs	Updates scope soft-assert expectations to the new structured `AreEqual` message format.
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreEqualTests.cs	Rewrites `AreEqual`/`AreNotEqual` failure tests to assert full structured messages; adds regressions for metadata + multiline placeholders.
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hant.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hans.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.tr.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ru.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pt-BR.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pl.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ko.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ja.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.it.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.fr.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.es.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.de.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.cs.xlf	Adds new structured-message summary keys (new state).
src/TestFramework/TestFramework/Resources/FrameworkMessages.resx	Introduces new summary strings for `AreEqual` / `AreNotEqual` structured messages.
src/TestFramework/TestFramework/Assertions/Assert.cs	Adds a binary call-site formatting helper supporting multiline placeholders.
src/TestFramework/TestFramework/Assertions/Assert.AreEqual.String.cs	Routes string `AreNotEqual` failures through the new structured message path.
src/TestFramework/TestFramework/Assertions/Assert.AreEqual.InterpolatedStringHandlers.cs	Updates interpolated-string handler failure paths to pass caller expressions into the new structured reporting.
src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs	Implements structured-message formatting for `AreEqual`/`AreNotEqual` (generic/object paths), including evidence blocks and expected/actual metadata.
docs/RFCs/012-Structured-Assertion-Messages.md	Updates RFC examples/text to remove the “(case-sensitive)” suffix for string equality summaries.

Copilot's findings

Files reviewed: 21/21 changed files
Comments generated: 3

Evangelink

Review Summary — RFC 012: AreEqual / AreNotEqual structured messages

The overall approach is sound. The EvidenceBlock builder, FormatBinaryCallSiteExpression, multiline-expression placeholder, and ExpectedText/ActualText/Data["assert.*"] population all look correct. The test rewrites faithfully capture the new format.

Four findings worth addressing:

#	Severity	File	Summary
1	Medium	`Assert.AreEqual.String.cs`	Culture/ignoreCase overloads NOT migrated — still produce old `"Assert.AreEqual failed."` format, creating a visible format split within the same API family. At minimum needs a comment or RFC follow-up reference.
2	Low–Medium	`Assert.AreEqual.InterpolatedStringHandlers.cs`	Delta (float/double/decimal/long) `AreNotEqual` paths still use `BuildUserMessageForNotExpectedExpressionAndActualExpression` (old format) while non-delta uses structured — inconsistency within the same struct.
3	Low	`Assert.AreEqual.cs` line 264	`diffIndex` is computed but silently dropped for different-length strings; RFC doc example shows it should appear. Either update the doc or include the index in the message.
4	Low	`AssertTests.AreEqualTests.cs`	`AreEqualLongStringsShowsFullStrings` acknowledges removed truncation but cites no tracking reference. Long-string truncation is a genuine usability regression; a linked issue would prevent it from going stale.

No blocking correctness bugs found. The public API additions (ExpectedText, ActualText, resx keys, FormatBinaryCallSiteExpression) follow existing conventions. XLF files were auto-generated (not hand-edited). The [DoesNotReturn] annotation is preserved on both ReportAssert*Failed methods.

Generated by Expert Code Review (on open) for issue #8231 · ● 18.3M

…ession; clarify RFC case-sensitivity note

Evangelink · 2026-05-14T18:15:15Z

@copilot resolve the merge conflicts in this pull request

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

Copilot · 2026-05-14T18:22:53Z

@copilot resolve the merge conflicts in this pull request

Done. I merged origin/main, resolved the conflict in AssertTests.ScopeTests.cs, and pushed the merge commit 0cd4ef2.

# Conflicts: # test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs

Copilot

Copilot's findings

Files reviewed: 21/21 changed files
Comments generated: 3

…dings; extract AreEqual(1,2) test message into shared const; update OutputTests for structured AreEqual message Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Evangelink · 2026-05-16T10:56:03Z

@copilot resolve the merge conflicts in this pull request

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

Copilot · 2026-05-16T11:05:30Z

@copilot resolve the merge conflicts in this pull request

Resolved. I merged origin/main, fixed the conflicts in FrameworkMessages.resx, all FrameworkMessages.*.xlf files, and AssertTests.ScopeTests.cs, then pushed commits ee56476 and 3b1c88f.

Evangelink · 2026-05-16T14:27:13Z

@copilot resolve the merge conflicts in this pull request

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

Copilot · 2026-05-16T14:34:27Z

@copilot resolve the merge conflicts in this pull request

Resolved. I merged origin/main, fixed the conflict in src/TestFramework/TestFramework/Resources/FrameworkMessages.resx by keeping both the structured AreEqual/AreNotEqual summary keys and the incoming Matches/DoesNotMatch regex keys, and pushed merge commit 1950cf2d0.

Evangelink added 12 commits May 12, 2026 18:06

test: convert structured-message assertions to raw strings + full equ…

e90c90e

…ality

Address expert reviewer feedback (H1, M2, L3, N1)

62c39f5

Address expert reviewer iteration 2 (N1 IsNullOrEmpty, N2 nullable Ty…

1b7bc4a

…pe.FullName, L1+L3 regression tests)

Address expert reviewer iteration 3 (L1 rename test, L2 add AreNotEqu…

ae0742c

…al full-message test, N4 cache GetType, N5 hoist line break chars, N6 consolidate string branch)

Fix SA1311/IDE1006: rename s_lineBreakChars to LineBreakChars

46a0016

Copilot AI review requested due to automatic review settings May 14, 2026 17:04

Copilot started reviewing on behalf of Evangelink May 14, 2026 17:08 View session

Copilot AI reviewed May 14, 2026

View reviewed changes

Comment thread src/TestFramework/TestFramework/Assertions/Assert.cs Outdated

Comment thread docs/RFCs/012-Structured-Assertion-Messages.md Outdated

Comment thread docs/RFCs/012-Structured-Assertion-Messages.md

Evangelink commented May 14, 2026

View reviewed changes

Address PR review: use IsNullOrWhiteSpace in FormatBinaryCallSiteExpr…

af81537

…ession; clarify RFC case-sensitivity note

Copilot started work on behalf of Evangelink May 14, 2026 18:15 View session

Merge origin/main and resolve ScopeTests conflicts

0cd4ef2

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 14, 2026 18:22

Evangelink review requested due to automatic review settings May 14, 2026 18:22

Copilot finished work on behalf of Evangelink May 14, 2026 18:23

Merge branch 'main' into dev/amauryleve/structured-messages-equality

842f318

# Conflicts: # test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs

Copilot AI review requested due to automatic review settings May 14, 2026 19:21

Copilot AI reviewed May 14, 2026

View reviewed changes

Comment thread src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs

Comment thread docs/RFCs/012-Structured-Assertion-Messages.md Outdated

Comment thread test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs Outdated

Copilot started reviewing on behalf of Evangelink May 14, 2026 19:41 View session

Address PR review: drop `case-sensitive'' from RFC string section hea…

6ead0eb

…dings; extract AreEqual(1,2) test message into shared const; update OutputTests for structured AreEqual message Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot started work on behalf of Evangelink May 16, 2026 10:56 View session

Copilot AI and others added 2 commits May 16, 2026 11:02

Merge origin/main and resolve TestFramework conflicts

ee56476

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

Restore AreEqual summary xlf entries after merge conflict resolution

3b1c88f

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 16, 2026 11:04

Evangelink review requested due to automatic review settings May 16, 2026 11:04

Copilot finished work on behalf of Evangelink May 16, 2026 11:05

Copilot started work on behalf of Evangelink May 16, 2026 14:27 View session

Evangelink mentioned this pull request May 16, 2026

Fix TelemetryTests flake from parallel dotnet test races #8292

Open

Merge origin/main and resolve conflicts

1950cf2

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 16, 2026 14:34

Evangelink review requested due to automatic review settings May 16, 2026 14:34

Copilot finished work on behalf of Evangelink May 16, 2026 14:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apply structured assertion messages (RFC 012) to AreEqual / AreNotEqual#8231

Apply structured assertion messages (RFC 012) to AreEqual / AreNotEqual#8231
Evangelink wants to merge 19 commits into
mainfrom
dev/amauryleve/structured-messages-equality

Evangelink commented May 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Evangelink left a comment

Uh oh!

Evangelink commented May 14, 2026

Uh oh!

Copilot AI commented May 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Evangelink commented May 16, 2026

Uh oh!

Copilot AI commented May 16, 2026

Uh oh!

Evangelink commented May 16, 2026

Uh oh!

Copilot AI commented May 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Evangelink commented May 14, 2026

Summary

Output format

Notable design points

Tests

Out of scope (deferred to follow-up PRs)

Reviewer iterations

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Evangelink left a comment

Choose a reason for hiding this comment

Review Summary — RFC 012: AreEqual / AreNotEqual structured messages

Uh oh!

Evangelink commented May 14, 2026

Uh oh!

Copilot AI commented May 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Evangelink commented May 16, 2026

Uh oh!

Copilot AI commented May 16, 2026

Uh oh!

Evangelink commented May 16, 2026

Uh oh!

Copilot AI commented May 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants