Skip to content

Add regression-tracking benchmarks for common colour/interop paths#4476

Merged
mattleibow merged 1 commit into
mainfrom
mattleibow-expand-regression-benchmarks
Jul 17, 2026
Merged

Add regression-tracking benchmarks for common colour/interop paths#4476
mattleibow merged 1 commit into
mainfrom
mattleibow-expand-regression-benchmarks

Conversation

@mattleibow

@mattleibow mattleibow commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

What

Expand benchmarks/SkiaSharp.Benchmarks.Tracking with a small, curated set of permanent regression benchmarks, derived from the areas the recent perf/agentic PRs touch. Each new benchmark is one .cs file in Tracking/Benchmarks/, so it's automatically picked up by both the package-mode project and the source-mode …Tracking.Source project (the ⭐ pr column).

Following the project's philosophy (and how dotnet/performance / aspnetcore structure their suites), this is a scaled pyramid — a few small common-path trackers + one large "touch many things" indicator — not a micro-benchmark per PR.

Tier 1 — small common-path trackers

Tier 2 — large composite

Not added: #4372 (niche SKNWayCanvas/SKOverdrawCanvas debug wrappers) and #4468 (HarfBuzz — a separate assembly the tracking project doesn't reference).

Constraints honoured

  • Public API only (project references the nightly NuGet, no internal SkiaApi).
  • Deterministic + machine-independent: fixed seeds, no fonts/external content.
  • Never-rename names chosen for permanent history keys.
  • Compile against both the 3.119 baseline and 4.x nightly (only SKPath/SKPathBuilder needs the existing #if).

Validation

Local (short job)

Built against SkiaSharp 4.151.0-preview.2.1 and 3.119.4; a --job short run emits time + allocation data for every case:

Benchmark Mean Allocated
ColorMathBenchmark.ToColorF (managed, #4370) 9,986 ns 0 B
ColorMathBenchmark.ToColor (native reverse, sentinel) 15,930 ns 0 B
ColorMathBenchmark.PreMultiply (#4385) 10,239 ns 0 B
ColorMathBenchmark.UnPreMultiply (#4385) 6,986 ns 0 B
RasterImageLifecycleBenchmark.CreateRasterImage (#4455) 235 µs 55,296 B
RasterImageLifecycleBenchmark.CreateDataWithReleaseProc (#4453) 57.7 µs 34,816 B
RuntimeEffectShaderBenchmark.DrawFrame (#4442) 328 µs 680 B
SceneRenderBenchmark.RenderFrame (Complexity 1 / 4) 1.05 ms / 4.53 ms 0 B

(The managed ToColorF already measuring faster than the still-native ToColor is the #4370 win showing through.)

CI baseline — pr column, run 29608575980 (2026-07-17)

Full source build of this branch (the ⭐ pr role) on the GitHub-hosted runners — one point per OS, the baseline each future run diffs against. Managed allocation is OS-independent (single column). On pr/HEAD ToColorF is still native, so expect its line to drop once #4370 merges.

Benchmark (params) Linux Windows macOS Allocated
ColorMathBenchmark.ToColorF (4096) 21.5 µs 32.6 µs 14.5 µs 0 B
ColorMathBenchmark.ToColor (4096) 14.1 µs 15.8 µs 21.4 µs 0 B
ColorMathBenchmark.PreMultiply (4096) 15.9 µs 23.1 µs 11.1 µs 0 B
ColorMathBenchmark.UnPreMultiply (4096) 14.6 µs 19.2 µs 9.0 µs 0 B
RasterImageLifecycleBenchmark.CreateRasterImage (256) 263.5 µs 283.7 µs 166.3 µs 54.0 KB
RasterImageLifecycleBenchmark.CreateDataWithReleaseProc (256) 117.2 µs 131.1 µs 84.6 µs 34.0 KB
RuntimeEffectShaderBenchmark.DrawFrame 2.83 ms 1.12 ms 558 µs ~0.7 KB
SceneRenderBenchmark.RenderFrame (Complexity 1) 4.90 ms 2.56 ms 1.46 ms 0 B
SceneRenderBenchmark.RenderFrame (Complexity 4) 19.18 ms 10.76 ms 6.04 ms 0 B

All 21 legs green (3 OSes × source pr + 5 package roles incl. prev-major 3.119.4); the perf-dashboard artifact renders all four benchmarks.

Absolute ns are environment-dependent; the tracked signals are per-OS version-role ratios + allocations.

Expand SkiaSharp.Benchmarks.Tracking with a small, curated set of permanent
regression benchmarks derived from recent perf/agentic PRs, following the
project's scaled-pyramid philosophy (a few small common-path trackers + one
large "touch many things" indicator, not per-PR micro sprawl).

Tier 1 - small common-path trackers:
- ColorMathBenchmark: managed SKColor->SKColorF (#4370) + SKPMColor
  PreMultiply/UnPreMultiply (#4385). Zero-alloc; the ToColorF/ToColor ratio
  (reverse stays native) is a managed-regression signal.
- RasterImageLifecycleBenchmark: create+dispose churn of SKImage.Create raster
  (#4455) and SKData.Create with a managed release proc (#4453). Allocation is
  the tracked signal for the leak-fix area.
- RuntimeEffectShaderBenchmark: per-frame animated SkSL shader - build uniforms
  (SKColor uniform hits #4370), ToShader, draw. Exercises the uniforms
  lifecycle whose SKData leak #4442 fixed.

Tier 2 - large composite:
- SceneRenderBenchmark: one broad frame (transforms, linear+radial gradients,
  filled/stroked primitives, a built path, a scaled image draw, a colour-filter
  layer, clipping, managed colour maths) so many future optimizations each nudge
  it - a merge indicator.

All are public-API only, deterministic (fixed seeds, no fonts/external content),
and compile against both the 3.119 baseline and 4.x nightly. Verified locally
against SkiaSharp 4.151 and 3.119.4; a short run emits time + allocation data for
every case.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ca69afac-998b-493d-a03c-d9b25dd3f15b
@github-actions

Copy link
Copy Markdown
Contributor

📦 Try the packages from this PR

Warning

Do not run these scripts without first reviewing the code in this PR.

Step 1 — Download the packages

bash / macOS / Linux:

curl -fsSL https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.sh | bash -s -- 4476

PowerShell / Windows:

iex "& { $(irm https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.ps1) } 4476"

Step 2 — Add the local NuGet source

dotnet nuget add source ~/.skiasharp/hives/pr-4476/packages --name skiasharp-pr-4476
More options
Option Description
--successful-only / -SuccessfulOnly Only use successful builds
--force / -Force Overwrite previously downloaded packages
--list / -List List available artifacts without downloading
--build-id ID / -BuildId ID Download from a specific build

Or download manually from Azure Pipelines — look for the nuget artifact on the build for this PR.

Remove the source when you're done:

dotnet nuget remove source skiasharp-pr-4476

@mattleibow
mattleibow merged commit d7eb05c into main Jul 17, 2026
27 of 85 checks passed
@mattleibow
mattleibow deleted the mattleibow-expand-regression-benchmarks branch July 17, 2026 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant