Skip to content

refactor: assert diagnostics by wildcard so failures show the actual output#114

Merged
vbreuss merged 1 commit into
mainfrom
test/diagnostic-assertions-show-actual-output
Jul 17, 2026
Merged

refactor: assert diagnostics by wildcard so failures show the actual output#114
vbreuss merged 1 commit into
mainfrom
test/diagnostic-assertions-show-actual-output

Conversation

@vbreuss

@vbreuss vbreuss commented Jul 17, 2026

Copy link
Copy Markdown
Member

await That(result.Diagnostics.Any(d => d.Contains("AWT101"))).IsTrue() collapses the diagnostics to a bool before the assertion sees them, so a failure reports only expected true, but it was false and says nothing about what the generator actually produced. Asserting on the collection instead keeps the diagnostics in the failure message:

Expected that result.Diagnostics
contains "*AWT999*" as wildcard at least once,
but it did not contain it

Collection:
[
  "(9,2): error AWT101: 'MyCode.Service' cannot be resolved: 'MyCode.Service' requires 'MyCode.IMissing…"
]

Converts all 192 sites across 55 files: .IsTrue() becomes Contains("*AWT101*").AsWildcard() and .IsFalse() becomes DoesNotContain("*AWT101*").AsWildcard().

Two shapes needed more than a mechanical rewrite. Three predicates combining terms with && flatten to an ordered wildcard (d.Contains("AWT120") && d.Contains("Repository") && d.Contains("Connection") becomes "*AWT120*Repository*Connection*"); && is order-independent and a wildcard is not, so these tightened slightly, and they pass. Three .IsFalse() predicates combining terms with || split into two DoesNotContain assertions each, since negating a disjunction distributes over both terms.

No behavioral change: the same 586 tests pass before and after.

`await That(result.Diagnostics.Any(d => d.Contains("AWT101"))).IsTrue()` collapses the diagnostics to a bool before the assertion sees them, so a failure reports only `expected true, but it was false` and says nothing about what the generator actually produced. Asserting on the collection instead keeps the diagnostics in the failure message:

```
Expected that result.Diagnostics
contains "*AWT999*" as wildcard at least once,
but it did not contain it

Collection:
[
  "(9,2): error AWT101: 'MyCode.Service' cannot be resolved: 'MyCode.Service' requires 'MyCode.IMissing…"
]
```

Converts all 192 sites across 55 files: `.IsTrue()` becomes `Contains("*AWT101*").AsWildcard()` and `.IsFalse()` becomes `DoesNotContain("*AWT101*").AsWildcard()`.

Two shapes needed more than a mechanical rewrite. Three predicates combining terms with `&&` flatten to an ordered wildcard (`d.Contains("AWT120") && d.Contains("Repository") && d.Contains("Connection")` becomes `"*AWT120*Repository*Connection*"`); `&&` is order-independent and a wildcard is not, so these tightened slightly, and they pass. Three `.IsFalse()` predicates combining terms with `||` split into two `DoesNotContain` assertions each, since negating a disjunction distributes over both terms.

No behavioral change: the same 586 tests pass before and after.
@vbreuss vbreuss self-assigned this Jul 17, 2026
@vbreuss
vbreuss enabled auto-merge (squash) July 17, 2026 08:19
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

Test Results

   18 files  ±0     18 suites  ±0   7m 16s ⏱️ -43s
1 128 tests ±0  1 127 ✅ ±0  1 💤 ±0  0 ❌ ±0 
5 501 runs  ±0  5 500 ✅ ±0  1 💤 ±0  0 ❌ ±0 

Results for commit 5efc38a. ± Comparison against base commit a9ecf93.

@vbreuss
vbreuss merged commit bc305b8 into main Jul 17, 2026
14 checks passed
@vbreuss
vbreuss deleted the test/diagnostic-assertions-show-actual-output branch July 17, 2026 08:26
@github-actions

Copy link
Copy Markdown

🚀 Benchmark Results

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 7763 2.45GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.302
[Host] : .NET 10.0.10 (10.0.10, 10.0.1026.32716), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Build Size Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 8 18.07 ns 0.481 ns 0.450 ns 1.08 144 B 1.00
Awaiten 8 16.676 ns 0.2029 ns 0.1798 ns 1.00 144 B 1.00
MsDI 8 1,445.674 ns 13.7127 ns 12.8269 ns 86.70 5688 B 39.50
Autofac 8 30,117.645 ns 249.7310 ns 233.5985 ns 1,806.20 33098 B 229.85
Jab 8 8.480 ns 0.0514 ns 0.0455 ns 0.51 96 B 0.67
PureDI 8 14.739 ns 0.1874 ns 0.1753 ns 0.88 128 B 0.89
DryIoc 8 701.761 ns 3.7910 ns 3.3606 ns 42.09 1528 B 10.61
SimpleInjector 8 11,382.468 ns 52.7675 ns 46.7770 ns 682.63 24760 B 171.94
baseline* 256 94.89 ns 2.127 ns 1.886 ns 1.09 2128 B 1.00
Awaiten 256 87.066 ns 0.4879 ns 0.4325 ns 1.00 2128 B 1.00
MsDI 256 14,300.002 ns 73.5324 ns 68.7822 ns 164.25 61016 B 28.67
Autofac 256 735,625.153 ns 2,368.8097 ns 2,099.8881 ns 8,449.21 738970 B 347.26
Jab 256 72.860 ns 0.7990 ns 0.7083 ns 0.84 2080 B 0.98
PureDI 256 87.162 ns 0.5644 ns 0.5004 ns 1.00 2112 B 0.99
DryIoc 256 43,124.942 ns 243.3957 ns 227.6725 ns 495.32 80640 B 37.89
SimpleInjector 256 345,076.086 ns 1,671.2733 ns 1,563.3101 ns 3,963.46 573141 B 269.33
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 9V74 2.60GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.302
[Host] : .NET 10.0.10 (10.0.10, 10.0.1026.32716), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Realistic Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 248.5 ns 3.15 ns 2.94 ns 0.84 568 B 1.00
Awaiten 297.1 ns 3.39 ns 3.18 ns 1.00 568 B 1.00
MsDI 666.6 ns 15.56 ns 14.55 ns 2.24 1104 B 1.94
Autofac 6,577.6 ns 45.84 ns 40.63 ns 22.14 10784 B 18.99
Jab 228.1 ns 2.15 ns 2.01 ns 0.77 432 B 0.76
DryIoc 472.8 ns 4.41 ns 3.91 ns 1.59 944 B 1.66
SimpleInjector 774.5 ns 8.95 ns 8.37 ns 2.61 1096 B 1.93
PureDI 238.4 ns 1.26 ns 1.18 ns 0.80 632 B 1.11
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 7763 2.62GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.302
[Host] : .NET 10.0.10 (10.0.10, 10.0.1026.32716), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Resolve Size Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 8 6.984 ns 0.0079 ns 0.0066 ns 1.00 - NA
Awaiten 8 7.005 ns 0.0046 ns 0.0041 ns 1.00 - NA
MsDI 8 7.271 ns 0.0230 ns 0.0192 ns 1.04 - NA
Autofac 8 124.863 ns 0.3956 ns 0.3304 ns 17.83 656 B NA
Jab 8 2.639 ns 0.0298 ns 0.0279 ns 0.38 - NA
PureDI 8 5.462 ns 0.0055 ns 0.0043 ns 0.78 - NA
DryIoc 8 8.716 ns 0.0065 ns 0.0051 ns 1.24 - NA
SimpleInjector 8 10.474 ns 0.0184 ns 0.0172 ns 1.50 - NA
baseline* 256 7.817 ns 0.0105 ns 0.0088 ns 1.01 - NA
Awaiten 256 7.771 ns 0.0098 ns 0.0082 ns 1.00 - NA
MsDI 256 7.286 ns 0.0096 ns 0.0085 ns 0.94 - NA
Autofac 256 123.289 ns 0.2610 ns 0.2180 ns 15.87 656 B NA
Jab 256 42.703 ns 0.0737 ns 0.0654 ns 5.50 - NA
PureDI 256 8.857 ns 0.0068 ns 0.0057 ns 1.14 - NA
DryIoc 256 8.748 ns 0.0058 ns 0.0051 ns 1.13 - NA
SimpleInjector 256 14.227 ns 0.0329 ns 0.0292 ns 1.83 - NA

baseline* rows show the corresponding Awaiten benchmark from the most recent successful main branch build with results, for regression comparison.

github-actions Bot added a commit that referenced this pull request Jul 17, 2026
…d so failures show the actual output (#114) by Valentin Breuß
github-actions Bot added a commit that referenced this pull request Jul 17, 2026
…d so failures show the actual output (#114) by Valentin Breuß
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant