Skip to content

refactor: enhance benchmark table display names#746

Merged
vbreuss merged 1 commit intomainfrom
topic/refactor-enhance-benchmark-table-display-names
May 1, 2026
Merged

refactor: enhance benchmark table display names#746
vbreuss merged 1 commit intomainfrom
topic/refactor-enhance-benchmark-table-display-names

Conversation

@vbreuss
Copy link
Copy Markdown
Member

@vbreuss vbreuss commented May 1, 2026

This pull request refactors and improves the logic for generating benchmark comparison comments in Pipeline/Build.Benchmarks.cs. The main focus is on restructuring how benchmark tables are parsed, buffered, and rendered, especially to support clearer baseline injection and improved table formatting.

Benchmark table processing and formatting improvements:

  • Introduced a TableRow record and a table buffer to accumulate table rows before rendering, allowing for more flexible processing and baseline injection. The logic for flushing the buffer and injecting baselines has been moved to a new FlushTableBuffer method.
  • Added logic to detect and strip common prefixes from benchmark row names, improving table readability. This is handled by the new FindCommonRowPrefix and ApplyHeaderAndPrefixStripping helper methods.
  • The summary comment now includes an explanatory note about baseline* rows only if a baseline was actually injected, making the output more relevant and less cluttered.
  • Improved the logic for identifying and bolding Mockolate rows, ensuring only relevant rows are highlighted and displayed in the correct format.

These changes collectively make the benchmark result comments more accurate, easier to read, and better structured for regression comparison.

@vbreuss vbreuss self-assigned this May 1, 2026
Copilot AI review requested due to automatic review settings May 1, 2026 13:47
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the benchmark PR-comment formatter in the Nuke build pipeline to improve table display names by buffering parsed table rows and applying common-prefix stripping before rendering (while preserving baseline injection and Mockolate-row emphasis).

Changes:

  • Buffer parsed markdown table rows and flush them as a unit at code-fence boundaries / non-table lines.
  • Add common-prefix detection and prefix stripping to shorten benchmark display names in the first column.
  • Keep baseline-row injection and Mockolate-row bolding, but apply them during the buffered flush pass.

Comment thread Pipeline/Build.Benchmarks.cs
Comment thread Pipeline/Build.Benchmarks.cs
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 1, 2026

@vbreuss vbreuss merged commit 381840a into main May 1, 2026
19 checks passed
@vbreuss vbreuss deleted the topic/refactor-enhance-benchmark-table-display-names branch May 1, 2026 13:54
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

🚀 Benchmark Results

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Intel Xeon Platinum 8370C CPU 2.80GHz (Max: 3.38GHz), 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.203
[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v4

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

CreateMock Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 206.44 ns 1.574 ns 1.314 ns 1.00 1048 B 1.00
Mockolate 206.59 ns 3.820 ns 3.573 ns 1.00 1048 B 1.00
Imposter 310.21 ns 7.441 ns 6.960 ns 1.50 2248 B 2.15
TUnitMocks 47.29 ns 1.408 ns 1.317 ns 0.23 224 B 0.21
Moq 1,427.94 ns 7.572 ns 7.083 ns 6.91 2096 B 2.00
NSubstitute 2,014.59 ns 28.828 ns 26.966 ns 9.75 5048 B 4.82
FakeItEasy 1,706.02 ns 17.248 ns 16.134 ns 8.26 2772 B 2.65
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 9V74 2.60GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.203
[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Event Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 311.5 ns 7.36 ns 6.89 ns 1.00 1.83 KB 1.00
Mockolate 312.8 ns 2.59 ns 2.43 ns 1.00 1.83 KB 1.00
Moq 13,671.3 ns 95.32 ns 84.50 ns 43.71 12.51 KB 6.84
NSubstitute 5,171.7 ns 32.78 ns 29.06 ns 16.54 9.05 KB 4.95
FakeItEasy 226,977.9 ns 462.67 ns 410.14 ns 725.74 15.39 KB 8.42
Imposter 1,361.0 ns 5.48 ns 4.28 ns 4.35 8.8 KB 4.82
TUnitMocks 174.1 ns 0.96 ns 0.90 ns 0.56 1.37 KB 0.75
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 7763 2.82GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.203
[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Method N Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 1 842.5 ns 21.78 ns 19.31 ns 1.00 3.9 KB 1.00
Indexer_Mockolate 1 1.031 μs 0.0299 μs 0.0280 μs 1.00 3.9 KB 1.00
Indexer_Moq 1 218.756 μs 1.4175 μs 1.2566 μs 212.30 20.37 KB 5.23
Indexer_NSubstitute 1 9.307 μs 0.1002 μs 0.0937 μs 9.03 12.84 KB 3.29
Indexer_FakeItEasy 1 12.137 μs 0.1197 μs 0.1061 μs 11.78 13.88 KB 3.56
Indexer_Imposter 1 1.057 μs 0.0419 μs 0.0392 μs 1.03 5.16 KB 1.32

|
| baseline* | 10 | 1,958.2 ns | 25.81 ns | 24.15 ns | 1.00 | 4.95 KB | 1.00 |
| Indexer_Mockolate | 10 | 2.684 μs | 0.0523 μs | 0.0489 μs | 1.00 | 4.95 KB | 1.00 |
| Indexer_Moq | 10 | 232.812 μs | 0.6529 μs | 0.5788 μs | 86.78 | 29.89 KB | 6.04 |
| Indexer_NSubstitute | 10 | 23.634 μs | 0.1439 μs | 0.1346 μs | 8.81 | 25.63 KB | 5.18 |
| Indexer_FakeItEasy | 10 | 26.053 μs | 0.2309 μs | 0.2047 μs | 9.71 | 35.47 KB | 7.16 |
| Indexer_Imposter | 10 | 2.311 μs | 0.0729 μs | 0.0682 μs | 0.86 | 7.97 KB | 1.61 |

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 7763 2.45GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.203
[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Method N Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 1 427.8 ns 4.98 ns 4.65 ns 1.00 2.15 KB 1.00
Method_Mockolate 1 489.3 ns 9.44 ns 8.83 ns 1.00 2.15 KB 1.00
Method_Moq 1 181,860.7 ns 826.40 ns 732.58 ns 371.79 14.58 KB 6.78
Method_NSubstitute 1 5,852.2 ns 83.36 ns 77.98 ns 11.96 9.06 KB 4.22
Method_FakeItEasy 1 6,321.9 ns 72.17 ns 67.51 ns 12.92 8.11 KB 3.78
Method_Imposter 1 621.2 ns 18.47 ns 17.28 ns 1.27 4.04 KB 1.88
Method_TUnitMocks 1 700.0 ns 28.20 ns 26.37 ns 1.43 2.9 KB 1.35

|
| baseline* | 10 | 710.7 ns | 16.66 ns | 15.59 ns | 1.00 | 2.36 KB | 1.00 |
| Method_Mockolate | 10 | 726.9 ns | 21.39 ns | 20.01 ns | 1.00 | 2.36 KB | 1.00 |
| Method_Moq | 10 | 188,686.4 ns | 1,284.34 ns | 1,201.38 ns | 259.75 | 18.48 KB | 7.83 |
| Method_NSubstitute | 10 | 9,103.4 ns | 121.99 ns | 101.87 ns | 12.53 | 11.52 KB | 4.88 |
| Method_FakeItEasy | 10 | 9,665.2 ns | 159.55 ns | 149.24 ns | 13.31 | 16.05 KB | 6.80 |
| Method_Imposter | 10 | 1,283.7 ns | 39.68 ns | 37.12 ns | 1.77 | 5.52 KB | 2.34 |
| Method_TUnitMocks | 10 | 1,678.5 ns | 34.88 ns | 29.12 ns | 2.31 | 4.49 KB | 1.90 |

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Intel Xeon Platinum 8370C CPU 2.80GHz (Max: 3.39GHz), 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.203
[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v4

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Method N Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 1 537.5 ns 14.45 ns 13.51 ns 1.00 2.46 KB 1.00
Property_Mockolate 1 577.9 ns 8.36 ns 7.41 ns 1.00 2.46 KB 1.00
Property_Moq 1 11,781.4 ns 19.65 ns 17.42 ns 20.39 10.39 KB 4.22
Property_NSubstitute 1 7,496.7 ns 35.94 ns 31.86 ns 12.97 11.45 KB 4.65
Property_FakeItEasy 1 8,410.2 ns 38.61 ns 36.12 ns 14.55 11.25 KB 4.57
Property_Imposter 1 483.2 ns 7.56 ns 7.07 ns 0.84 3.13 KB 1.27
Property_TUnitMocks 1 824.0 ns 5.52 ns 4.31 ns 1.43 2.51 KB 1.02

|
| baseline* | 10 | 1,030.7 ns | 4.85 ns | 4.30 ns | 1.00 | 2.95 KB | 1.00 |
| Property_Mockolate | 10 | 1,321.9 ns | 8.47 ns | 7.92 ns | 1.00 | 2.95 KB | 1.00 |
| Property_Moq | 10 | 18,550.4 ns | 36.59 ns | 34.22 ns | 14.03 | 18.28 KB | 6.19 |
| Property_NSubstitute | 10 | 16,776.2 ns | 17.24 ns | 16.12 ns | 12.69 | 21.08 KB | 7.14 |
| Property_FakeItEasy | 10 | 19,338.8 ns | 93.84 ns | 87.77 ns | 14.63 | 30.8 KB | 10.43 |
| Property_Imposter | 10 | 1,369.2 ns | 26.15 ns | 24.46 ns | 1.04 | 4.67 KB | 1.58 |
| Property_TUnitMocks | 10 | 2,767.0 ns | 15.83 ns | 14.80 ns | 2.09 | 4.66 KB | 1.58 |

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 7763 2.45GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.203
[Host] : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Callback Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 392.6 ns 10.88 ns 10.17 ns 1.00 1.79 KB 1.00
Mockolate 382.7 ns 4.77 ns 4.23 ns 1.00 1.79 KB 1.00
Moq 95,786.4 ns 969.44 ns 859.39 ns 250.29 8.88 KB 4.97
NSubstitute 4,450.0 ns 32.62 ns 30.52 ns 11.63 7.74 KB 4.33
FakeItEasy 4,962.3 ns 76.03 ns 71.12 ns 12.97 6.81 KB 3.80
Imposter 424.4 ns 3.35 ns 2.80 ns 1.11 2.38 KB 1.33
TUnitMocks 624.3 ns 5.71 ns 5.34 ns 1.63 2.63 KB 1.47

baseline* rows show the corresponding Mockolate benchmark from the most recent successful main branch build with results, for regression comparison.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 2, 2026

This is addressed in release v3.1.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state: released The issue is released

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants