Skip to content

Add phpbench benchmarks with CI regression detection#51

Merged
veewee merged 2 commits intophp-soap:mainfrom
veewee:feature/phpbench-setup
Mar 26, 2026
Merged

Add phpbench benchmarks with CI regression detection#51
veewee merged 2 commits intophp-soap:mainfrom
veewee:feature/phpbench-setup

Conversation

@veewee
Copy link
Copy Markdown
Member

@veewee veewee commented Mar 26, 2026

Summary

  • Adds phpbench benchmark suite comparing php-soap/encoding against ext-soap (C extension)
  • Tests both literal and encoded binding modes, at x1 and x500 item scales
  • GitHub Actions workflow runs benchmarks on base + PR branch on the same runner, then diffs with --ref to surface regressions
  • Standalone profiling scripts in benchmarks/scripts/ for large payload throughput (50-500MB) and micro-profiling

Benchmark structure

EncodeBench                         DecodeBench
  benchLiteral_x1                     benchLiteral_x1
  benchLiteral_x500                   benchLiteral_x500
  benchEncoded_x1                     benchEncoded_x1
  benchEncoded_x500                   benchEncoded_x500
  benchExtSoapLiteral_x1              benchExtSoapLiteral_x1
  benchExtSoapLiteral_x500            benchExtSoapLiteral_x500
  benchExtSoapEncoded_x1              benchExtSoapEncoded_x1
  benchExtSoapEncoded_x500            benchExtSoapEncoded_x500

Groups: --group=literal, --group=encoded, --group=ext-soap, --group=x1, --group=x500

Usage

# Full suite
vendor/bin/phpbench run benchmarks/ --report=aggregate

# Just encode, literal mode
vendor/bin/phpbench run benchmarks/EncodeBench.php --group=literal --report=aggregate

# Store a run and compare later
vendor/bin/phpbench run benchmarks/ --store --tag=before --report=aggregate
# ... make changes ...
vendor/bin/phpbench run benchmarks/ --ref=before --report=aggregate

Test plan

  • vendor/bin/phpbench run benchmarks/ --report=aggregate completes with 16 subjects, 0 errors
  • --group=ext-soap filters to ext-soap benchmarks only
  • CI workflow runs on this PR and shows comparison output

veewee added 2 commits March 26, 2026 08:58
Benchmarks:
- EncodeBench / DecodeBench: php-soap/encoding vs ext-soap (C extension),
  literal vs encoded binding, x1 and x500 items
- Groups allow filtering: --group=literal, --group=ext-soap, --group=x500
- Standalone scripts in benchmarks/scripts/ for large payload throughput
  and detailed profiling

CI:
- GitHub Actions workflow runs benchmarks on both base and PR branch
  using the same runner, then compares with --ref for regression detection

Run: vendor/bin/phpbench run benchmarks/ --report=aggregate
@veewee veewee merged commit 9e3519e into php-soap:main Mar 26, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant