feat: add scripts/dev/test.sh — chunked FIFO local test wrapper (~1.75x faster)#1
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a Bash wrapper (scripts/dev/test.sh) that collects pytest node ids and dispatches them to pytest-xdist in fixed-size chunks with a resumable cursor at .pytest_cache/fast-test-cursor. Despite the PR description's claim of "Zero impact on pyproject.toml, CI workflows", the change also adds pytest-xdist as a required test extra and enables -n=auto --dist=worksteal in the project's default addopts, which affects every direct uv run pytest invocation — including the CI workflow and the contributor flow recommended in CONTRIBUTING.md / the PR template.
Changes:
- New
scripts/dev/test.shwrapper implementing chunked FIFO dispatch over xdist with--resume,--reset,--bench,--chunk-size, and--pass-through. pyproject.toml: addspytest-xdist>=3.5to thetestextra and-n=auto,--dist=workstealto defaultaddopts(affecting CI and all default local runs).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
scripts/dev/test.sh |
New chunked-FIFO pytest wrapper with cursor-based resume. |
pyproject.toml |
Adds pytest-xdist test dep and enables parallel xdist by default in pytest addopts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
52
to
+65
| @@ -60,6 +61,8 @@ addopts = [ | |||
| "-v", | |||
| "--strict-markers", | |||
| "--tb=short", | |||
| "-n=auto", | |||
| "--dist=worksteal", | |||
Comment on lines
+64
to
+65
| "-n=auto", | ||
| "--dist=worksteal", |
Comment on lines
+58
to
+66
| mapfile -t NODES < <( | ||
| uv run pytest -o addopts= --collect-only -q "${PASSTHROUGH[@]}" \ | ||
| 2>/dev/null | grep -E '::' || true | ||
| ) | ||
| TOTAL="${#NODES[@]}" | ||
| if (( TOTAL == 0 )); then | ||
| echo "[fast-test] no tests collected" >&2 | ||
| exit 1 | ||
| fi |
| set -euo pipefail | ||
|
|
||
| REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" | ||
| CURSOR_FILE="$REPO_ROOT/.pytest_cache/fast-test-cursor" |
Comment on lines
+37
to
+71
| --chunk-size) CHUNK_SIZE="$2"; shift 2 ;; | ||
| --resume) RESUME=1; shift ;; | ||
| --reset) RESET=1; shift ;; | ||
| --bench) BENCH=1; shift ;; | ||
| --) shift; PASSTHROUGH+=("$@"); break ;; | ||
| -h|--help) sed -n '2,22p' "$0"; exit 0 ;; | ||
| *) PASSTHROUGH+=("$1"); shift ;; | ||
| esac | ||
| done | ||
|
|
||
| if (( RESET )); then | ||
| rm -f "$CURSOR_FILE" | ||
| echo "[fast-test] cursor cleared" | ||
| exit 0 | ||
| fi | ||
|
|
||
| cd "$REPO_ROOT" | ||
| mkdir -p "$(dirname "$CURSOR_FILE")" | ||
|
|
||
| # 1. Collect node ids (override addopts so collection itself is serial/quiet). | ||
| echo "[fast-test] collecting tests ..." | ||
| mapfile -t NODES < <( | ||
| uv run pytest -o addopts= --collect-only -q "${PASSTHROUGH[@]}" \ | ||
| 2>/dev/null | grep -E '::' || true | ||
| ) | ||
| TOTAL="${#NODES[@]}" | ||
| if (( TOTAL == 0 )); then | ||
| echo "[fast-test] no tests collected" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # 2. Determine starting cursor. | ||
| START=0 | ||
| if (( RESUME )) && [[ -f "$CURSOR_FILE" ]]; then | ||
| START="$(cat "$CURSOR_FILE")" |
| START=0 | ||
| if (( RESUME )) && [[ -f "$CURSOR_FILE" ]]; then | ||
| START="$(cat "$CURSOR_FILE")" | ||
| echo "[fast-test] resuming from chunk cursor: test #$START" |
| --reset) RESET=1; shift ;; | ||
| --bench) BENCH=1; shift ;; | ||
| --) shift; PASSTHROUGH+=("$@"); break ;; | ||
| -h|--help) sed -n '2,22p' "$0"; exit 0 ;; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a single-file local DX wrapper,
scripts/dev/test.sh, that runs the existing pytest suite faster on developer machines without touching production code or CI.What it does
pytest --collect-only -q, then dispatches them in fixed-size chunks (default 200) throughpytest-xdistwith-n auto --dist=load..pytest_cache/fast-test-cursorafter every successful chunk, so a crash orCtrl-Ccan be resumed exactly where it left off with--resume.--chunk-size N,--reset,--bench, and--pass-through to pytest.Why this shape
--dist=load.uv run pytestentrypoint.pyproject.toml, CI workflows, or import-time behavior. Pure additive file underscripts/dev/.Measured impact (local, Windows + Git Bash, 16-core)
uv run pytest(serial)bash scripts/dev/test.sh --bench~1.75x speedup end-to-end with no production source changes.
Testing
uv run specify --helpuv sync && uv run pytestAdditional validation:
bash scripts/dev/test.sh --bench→ 1m 29.85s wallclock.--resumecontinued from exactly that point with only the remaining tests executed.--resetclears the cursor file;--chunk-sizeand--pass-through both exercised.AI Disclosure
GitHub Copilot (Claude) assisted with drafting
scripts/dev/test.sh, iterating on the chunked-FIFO dispatch design, and producing the benchmark comparison. All design decisions, validation runs, and the final wrapper shape were reviewed and accepted by the author.