Skip to content

#134 Better benchmarks across mappers with CI charts and gist history - #286

Merged
sanelli merged 11 commits into
mainfrom
134-better-benchmark
Jul 30, 2026
Merged

#134 Better benchmarks across mappers with CI charts and gist history#286
sanelli merged 11 commits into
mainfrom
134-better-benchmark

Conversation

@sanelli

@sanelli sanelli commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Summary

  • Expand Mappa.Benchmark into a broad AutoMapper / Mapster / Mapperly / Mappa suite (objects, enums, collections, memory, polymorphism, IQueryable, nested DTOs) with AutoMapper as the ratio baseline.
  • Rework RunBenchmarks.ps1 / SVG helpers to emit summary tables, TIME/MEMORY bar charts, and Mappa-only history rows; publish history + charts via UpdateBenchmarkGists.ps1.
  • Wire chart-subset benchmarks into PR and main CI, extend the coverage PR comment with benchmark summary/trends, and document the suite (including root README gist SVGs).

Test plan

  • .\Scripts\RunBenchmarks.ps1 -ChartBenchmarksOnly completes and writes .mappa-benchmark/ summary + SVG outputs
  • .\Scripts\UpdateBenchmarkGists.ps1 -DryRun merges gist history and regenerates history SVGs without upload
  • .\Scripts\CommentPullRequestCoverage.ps1 -DryRun includes Benchmarks + trends sections
  • PR CI runs chart-only benchmarks before the coverage comment
  • Confirm root README gist SVG links resolve once the gist files are populated on main

sanelli and others added 9 commits July 30, 2026 19:45
@sanelli @cursoragent
#134 Default RebuildUponVersionChange to Release and bump to alpha.25.
e3c525e 
Build all projects with -c Release so downstream packages and benchmarks use optimized output.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Expand Mappa.Benchmark suite across common mapper scenarios.
5990299 
Add class/record/struct, collection, nested DTO, memory, polymorphic, and IQueryable benchmarks with AutoMapper as baseline and ~300-entry collections including dictionaries.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Rework RunBenchmarks charts and fix AutoMapper profile loading.
b0562ed 
Use AddProfile per scenario, ConvertUsing for dictionaries, and emit summary/history/SVG outputs under .mappa-benchmark with Automapper, Mapster, Mapperly, Mappa bar order.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Plot mean time in microseconds with ListToArray in SVG charts.
5ceca2b 
Add ListToArrayBenchmark before FastListToArrayBenchmark and use a 50 us Y-axis tick on the mean-time chart.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Add benchmark PR trends and MAPPA-BENCHMARK-HISTORY gist updater.
f599bf9 
Extend the coverage PR comment with full mapper summary and three Mappa trends, and append history rows while tolerating a missing or header-only gist file.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Add multi-line benchmark history SVGs and publish summary charts…
33a3a8f 
… to gist.

Generate coverage-style TIME/MEMORY history charts for the shared benchmark subset, rename summary SVG outputs, and upload all four SVGs from UpdateBenchmarkGists.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Wire chart-only benchmarks into PR and main workflows.
edcda77 
Add -ChartBenchmarksOnly to RunBenchmarks and run that subset in pr.yml before the coverage comment and in merge-main.yml before UpdateBenchmarkGists.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Document benchmarks and embed gist SVG charts in the root README.
1cacbff 
Expand development.md and add Mappa.Benchmark/README.md; append TIME/MEMORY summary and history SVGs to the repository README.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Add benchmark and pre-PR dependency checks to workflow rules.
da54620 
Require evaluating new Mappa.Benchmark scenarios for features, keep new benches out of the SVG chart subset unless asked, and ask before opening dependency-update issues.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli sanelli linked an issue Jul 30, 2026 that may be closed by this pull request
Better benchmark #134
Closed
@sanelli sanelli mentioned this pull request Jul 30, 2026
Closed
@github-actions

github-actions Bot commented Jul 30, 2026 •
edited
Loading

Copy link
Copy Markdown

Code coverage

Compared against the latest published coverage history from the code coverage gist.

Metric Baseline PR Δ Trend
Line 98.4% 98.4% 0 =
Branch 90.6% 90.6% 0 =
Method 100% 100% 0 =

No coverage metric dropped by more than 1 percentage point(s) versus baseline.

Benchmarks

AutoMapper is the BenchmarkDotNet ratio baseline. Absolute values are mean time in nanoseconds and allocated bytes. Ratio columns are competitor / Mappa as a percentage (n/a when either side is missing or Mappa is zero).

Benchmark Automapper Mean (ns) Automapper Allocated (B) Mapster Mean (ns) Mapster Allocated (B) Mapperly Mean (ns) Mapperly Allocated (B) Mappa Mean (ns) Mappa Allocated (B) Automapper/Mappa Time Automapper/Mappa Alloc Mapperly/Mappa Time Mapperly/Mappa Alloc Mapster/Mappa Time Mapster/Mappa Alloc
ArrayToListBenchmark 67.84 32 73923.91 194456 7025.83 38456 6970.15 38456 1% 0.1% 100.8% 100% 1060.6% 505.7%
DictionaryBenchmark 93884 185221.12 82740 214364.16 9955 46018.56 11928 58316.8 787.1% 317.6% 83.5% 78.9% 693.7% 367.6%
FastListToArrayBenchmark n/a n/a 74150 194426.88 7664 38420.48 7266 38420.48 n/a n/a 105.5% 100% 1020.5% 506.1%
IQueryableProjectionBenchmark 622800 31784.96 603700 31016.96 621100 32983.04 616900 33105.92 101% 96% 100.7% 99.6% 97.9% 93.7%
ListToArrayBenchmark n/a n/a 71161 194426.88 6842 38420.48 6884 38420.48 n/a n/a 99.4% 100% 1033.7% 506.1%
ListToHashSetBenchmark 249.2 5765.12 2194.7 5765.12 2009.2 5806.08 1463.4 5765.12 17% 100% 137.3% 100.7% 150% 100%
NestedDtoBenchmark 82156 225320.96 91005 227471.36 7411 38942.72 7593 38942.72 1082% 578.6% 97.6% 100% 1198.5% 584.1%

Benchmark trends (Mappa)

Lower is better. Compared against the latest published Mappa-only rows from MAPPA-BENCHMARK-HISTORY.md.

Benchmark Measure Baseline PR Δ Trend
ArrayToListBenchmark TIME_NS 3140.81 6970.15 +3829.34 ▼
ArrayToListBenchmark ALLOC_B 38456 38456 0 =
DictionaryBenchmark TIME_NS 6678 11928 +5250 ▼
DictionaryBenchmark ALLOC_B 58316.8 58316.8 0 =
ListToArrayBenchmark TIME_NS 3169 6884 +3715 ▼
ListToArrayBenchmark ALLOC_B 38420.48 38420.48 0 =
FastListToArrayBenchmark TIME_NS 3252 7266 +4014 ▼
FastListToArrayBenchmark ALLOC_B 38420.48 38420.48 0 =
IQueryableProjectionBenchmark TIME_NS 161700 616900 +455200 ▼
IQueryableProjectionBenchmark ALLOC_B 32983.04 33105.92 +122.88 ▼
NestedDtoBenchmark TIME_NS 3329 7593 +4264 ▼
NestedDtoBenchmark ALLOC_B 38942.72 38942.72 0 =
ListToHashSetBenchmark TIME_NS 672.2 1463.4 +791.2 ▼
ListToHashSetBenchmark ALLOC_B 5765.12 5765.12 0 =

sanelli and others added 2 commits July 30, 2026 22:22
@sanelli @cursoragent
#134 Add competitor/Mappa percentage ratios and align PR trends with …
dfde0bb 
…SVG benches.

Extend the benchmark summary with Automapper/Mapperly/Mapster over Mappa percentage columns, point PR trends at BenchmarkChartNames, and clarify the outdated-package pre-PR check command.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli @cursoragent
#134 Add competitor/Mappa percentage bar charts to benchmarks and gist.
ecedecd 
Generate TIME/MEMORY percentage SVGs for the shared chart subset, publish them from UpdateBenchmarkGists, and embed them in the root README.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sanelli
sanelli merged commit 51951a4 into main Jul 30, 2026
1 check passed
@sanelli
sanelli deleted the 134-better-benchmark branch July 30, 2026 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Better benchmark

1 participant

@sanelli