Skip to content

chore(bench): wire make bench, add pytest-benchmark, record baseline#334

Merged
plind-junior merged 2 commits into
testfrom
chore/benchmarks
Jul 2, 2026
Merged

chore(bench): wire make bench, add pytest-benchmark, record baseline#334
plind-junior merged 2 commits into
testfrom
chore/benchmarks

Conversation

@plind-junior

Copy link
Copy Markdown
Collaborator

the benchmark suite was implemented but unrunnable out of the box: pytest-benchmark wasn't in the [dev] extras, and the bench_*.py filenames don't match pytest's default python_files glob, so the documented invocation collected zero tests.

  • add pytest-benchmark to the [dev] optional-dependencies
  • add a make bench target that folds in the python_files override
  • record the first baseline (medians at 1k/10k) in benchmarks/README.md and flip its status from "not implemented yet" to implemented
  • gitignore the bench.json / .benchmarks run artifacts

What changed

Why

What might break

VEP

Tests

  • make check passes locally (lint + mypy + pytest)
  • New / changed behaviour has a test
  • CHANGELOG.md updated under ## [Unreleased]

the benchmark suite was implemented but unrunnable out of the box:
pytest-benchmark wasn't in the [dev] extras, and the bench_*.py
filenames don't match pytest's default python_files glob, so the
documented invocation collected zero tests.

- add pytest-benchmark to the [dev] optional-dependencies
- add a `make bench` target that folds in the python_files override
- record the first baseline (medians at 1k/10k) in benchmarks/README.md
  and flip its status from "not implemented yet" to implemented
- gitignore the bench.json / .benchmarks run artifacts
Copilot AI review requested due to automatic review settings July 1, 2026 17:54
@github-actions github-actions Bot added packaging packaging, build metadata, and make targets size: S 50-199 changed non-doc lines labels Jul 1, 2026
@plind-junior plind-junior changed the base branch from main to test July 1, 2026 17:54
@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2ae3b3d8-ac91-4985-b18e-03a8dbd9792a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/benchmarks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes the existing benchmark suite runnable out of the box. Previously the suite couldn't be executed as documented: pytest-benchmark was missing from the [dev] extras, and the bench_*.py filenames don't match pytest's default python_files glob, so the documented invocation collected zero tests. The change wires up tooling and records a first baseline, fitting under the 0.3 "operational maturity" milestone in ROADMAP.md.

Changes:

  • Add pytest-benchmark>=5,<6 to the [dev] optional dependencies.
  • Add a make bench target (with the python_files collection override and JSON output), and update .PHONY, help text, and clean.
  • Update benchmarks/README.md to mark the suite implemented and record baseline medians; gitignore bench.json and .benchmarks/.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

File Description
pyproject.toml Adds pytest-benchmark to the [dev] extras so benchmarks install with dev setup.
Makefile Adds a bench target with the required python_files override, plus help/.PHONY/clean updates.
benchmarks/README.md Flips status to implemented and records a baseline; however the invocation section still describes the pre-PR state and contradicts these changes, and the recorded version (vouch 1.0.0) doesn't match the project version 0.0.1.
.gitignore Ignores benchmark run artifacts (bench.json, .benchmarks/).

The Makefile, pyproject.toml, and .gitignore changes are consistent with each other. The remaining concerns are documentation inconsistencies in benchmarks/README.md where the newly-added text still says pytest-benchmark isn't in [dev] and that make bench isn't wired (both fixed by this PR), and the baseline environment references a nonexistent vouch 1.0.0.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread benchmarks/README.md
is the bar this benchmark was written to guard.

Environment: 13th Gen Intel Core i9-13900K (16 threads), ~22 GB RAM,
Python 3.14, vouch 1.0.0. Full per-run detail (min/max/stddev, machine
Comment thread benchmarks/README.md
Comment on lines +66 to +69
pull them in. `pytest-benchmark` isn't in the `[dev]` extras, and the
`bench_*.py` filenames don't match pytest's default `python_files`
glob — so the invocation needs both an install and a collection
override:
Comment thread benchmarks/README.md
Comment on lines +78 to +79
`make bench` is not wired in the Makefile yet; when it is, it should
fold in the `python_files` override so this isn't a footgun.
@plind-junior

Copy link
Copy Markdown
Collaborator Author

@copilot resolve the merge conflicts in this pull request

Copilot AI commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Resolved by merging origin/test into this branch and fixing the Makefile conflict in commit dd6529b.

@plind-junior plind-junior merged commit 35eac96 into test Jul 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

packaging packaging, build metadata, and make targets size: S 50-199 changed non-doc lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants