chore(bench): wire make bench, add pytest-benchmark, record baseline#334
Conversation
the benchmark suite was implemented but unrunnable out of the box: pytest-benchmark wasn't in the [dev] extras, and the bench_*.py filenames don't match pytest's default python_files glob, so the documented invocation collected zero tests. - add pytest-benchmark to the [dev] optional-dependencies - add a `make bench` target that folds in the python_files override - record the first baseline (medians at 1k/10k) in benchmarks/README.md and flip its status from "not implemented yet" to implemented - gitignore the bench.json / .benchmarks run artifacts
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR makes the existing benchmark suite runnable out of the box. Previously the suite couldn't be executed as documented: pytest-benchmark was missing from the [dev] extras, and the bench_*.py filenames don't match pytest's default python_files glob, so the documented invocation collected zero tests. The change wires up tooling and records a first baseline, fitting under the 0.3 "operational maturity" milestone in ROADMAP.md.
Changes:
- Add
pytest-benchmark>=5,<6to the[dev]optional dependencies. - Add a
make benchtarget (with thepython_filescollection override and JSON output), and update.PHONY, help text, andclean. - Update
benchmarks/README.mdto mark the suite implemented and record baseline medians; gitignorebench.jsonand.benchmarks/.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
pyproject.toml |
Adds pytest-benchmark to the [dev] extras so benchmarks install with dev setup. |
Makefile |
Adds a bench target with the required python_files override, plus help/.PHONY/clean updates. |
benchmarks/README.md |
Flips status to implemented and records a baseline; however the invocation section still describes the pre-PR state and contradicts these changes, and the recorded version (vouch 1.0.0) doesn't match the project version 0.0.1. |
.gitignore |
Ignores benchmark run artifacts (bench.json, .benchmarks/). |
The Makefile, pyproject.toml, and .gitignore changes are consistent with each other. The remaining concerns are documentation inconsistencies in benchmarks/README.md where the newly-added text still says pytest-benchmark isn't in [dev] and that make bench isn't wired (both fixed by this PR), and the baseline environment references a nonexistent vouch 1.0.0.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| is the bar this benchmark was written to guard. | ||
|
|
||
| Environment: 13th Gen Intel Core i9-13900K (16 threads), ~22 GB RAM, | ||
| Python 3.14, vouch 1.0.0. Full per-run detail (min/max/stddev, machine |
| pull them in. `pytest-benchmark` isn't in the `[dev]` extras, and the | ||
| `bench_*.py` filenames don't match pytest's default `python_files` | ||
| glob — so the invocation needs both an install and a collection | ||
| override: |
| `make bench` is not wired in the Makefile yet; when it is, it should | ||
| fold in the `python_files` override so this isn't a footgun. |
|
@copilot resolve the merge conflicts in this pull request |
# Conflicts: # Makefile
Resolved by merging |
the benchmark suite was implemented but unrunnable out of the box: pytest-benchmark wasn't in the [dev] extras, and the bench_*.py filenames don't match pytest's default python_files glob, so the documented invocation collected zero tests.
make benchtarget that folds in the python_files overrideWhat changed
Why
What might break
VEP
Tests
make checkpasses locally (lint + mypy + pytest)CHANGELOG.mdupdated under## [Unreleased]