Skip to content

feat: add end-to-end benchmark for contact-book-manager app#89

Merged
akroshg merged 1 commit intomainfrom
akroshg/add-contact-book-benchmark
Mar 12, 2026
Merged

feat: add end-to-end benchmark for contact-book-manager app#89
akroshg merged 1 commit intomainfrom
akroshg/add-contact-book-benchmark

Conversation

@akroshg
Copy link
Copy Markdown
Contributor

@akroshg akroshg commented Mar 11, 2026

Add a criterion-based end-to-end benchmark that compiles the real contact-book-manager templates into a protocol binary and measures performance across three areas:

  • Protocol deserialization measures WebUIProtocol::from_protobuf() throughput
  • Handler rendering (no plugin) renders the dashboard page at 10 / 100 / 1,000 contacts
  • Handler rendering (FastHydration plugin) same with hydration markers enabled

Key design choices

  • Always fresh protocol templates are compiled via webui::build() at benchmark time from examples/app/contact-book-manager/src/. No cached binary template changes are automatically reflected in the next run.
  • Realistic state contacts are generated with names, emails, phones, companies, groups, favorites, and addresses matching the app's state.json schema.
  • Summary table a compact performance report (Iters, Avg, Min, Max, Dev%, P50, P90, P99, IQR, Bytes) is printed after criterion finishes for easy sharing.

Quality gate integration

  • cargo xtask check now includes a bench (validate) step that runs --test mode (compile + one iteration per benchmark) to catch breakage early.
  • cargo xtask bench webui runs the full contact-book benchmark with measurements.

Files changed

File Change
crates/webui/benches/contact_book_bench.rs New benchmark file (602 lines)
crates/webui/benches/README.md Usage docs, result interpretation, regression detection
crates/webui/Cargo.toml Add criterion, webui-handler, webui-protocol dev-deps + [[bench]]
xtask/src/main.rs Add bench validate step to check, add webui bench target
Cargo.lock Lockfile update

Running

# Quick validation
cargo bench -p webui --bench contact_book_bench -- --test

# Full benchmark
cargo bench -p webui --bench contact_book_bench

# Via xtask
cargo xtask bench webui

Add a criterion-based benchmark that compiles the real contact-book-manager
templates into a protocol binary and measures performance across three areas:

- Protocol deserialization (from_protobuf)
- Handler rendering without plugin at 10/100/1000 contacts
- Handler rendering with FastHydration plugin at 10/100/1000 contacts

The protocol is built from live source at benchmark time so any template
change is automatically reflected in the next run. A summary table with
Iters, Avg, Min, Max, Dev%, P50, P90, P99, IQR, and output Bytes is
printed after criterion finishes.

Also adds:
- Benchmark validation step to cargo xtask check (--test mode)
- 'webui' target to cargo xtask bench
- benches/README.md with usage, result interpretation, and regression
  detection guidance

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@akroshg
Copy link
Copy Markdown
Contributor Author

akroshg commented Mar 11, 2026

Output when run with command: cargo bench -p webui --bench contact_book_bench

============================= WebUI Contact Book — Performance Summary =============================
Story                     Iters   Avg(ms)       Min       Max   Dev%       P50       P90       P99       IQR     Bytes
----------------------------------------------------------------------------------------------------
ProtocolParse             56144      0.05      0.04      2.70  34.8%      0.04      0.04      0.07      0.00     28538
Render/10                  4783      0.63      0.61      1.29   6.3%      0.62      0.64      0.72      0.02     25960
Render/100                  620      4.84      4.70      7.14   4.0%      4.79      4.97      5.65      0.10     56397
Render/1000                  54     56.31     52.49     61.78   4.0%     56.39     59.16     61.10      3.60    362930
RenderFAST/10              4640      0.65      0.62      1.85   6.0%      0.64      0.67      0.80      0.03     31052
RenderFAST/100              606      4.96      4.84      8.69   5.3%      4.92      5.00      5.64      0.05     68149
RenderFAST/1000              52     57.83     52.92     65.56   4.9%     57.46     60.86     65.35      2.53    443082
====================================================================================================
IQR = P75 − P25 (lower means more consistent)
All times in milliseconds
====================================================================================================

@akroshg
Copy link
Copy Markdown
Contributor Author

akroshg commented Mar 11, 2026

Fixes #69

@akroshg akroshg requested a review from mohamedmansour March 11, 2026 23:16
@akroshg akroshg merged commit 5d0bc6b into main Mar 12, 2026
15 checks passed
@akroshg akroshg deleted the akroshg/add-contact-book-benchmark branch March 12, 2026 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants