Examples

Examples

The repository contains two deterministic workloads. They demonstrate behavior without turning machine-specific timings into flaky assertions.

Source directory: https://github.com/Butterski/pytest-poolwatch/tree/master/examples

PR #86 scheduler regression

pytest-asyncio-cooperative PR #86 fixes a refill loop that scheduled only one replacement after several active tasks completed together.

The workload forces one asyncio.wait(..., FIRST_COMPLETED) call to return three completed tasks while additional tests remain queued. This reproduces the exact condition rather than relying on event-loop luck.

Released pre-fix scheduler

uv run --no-dev --group demo pytest -p no:asyncio \
  examples/pr86/test_workload.py \
  --max-asyncio-tasks=4 \
  --poolwatch \
  --poolwatch-underfill-threshold=0.05 \
  --poolwatch-json=.poolwatch/pr86-before.json \
  --poolwatch-html=.poolwatch/pr86-before.html

uv run --no-dev python examples/pr86/assert_report.py \
  before .poolwatch/pr86-before.json

The demo dependency group pins released version 0.40.0, which predates the fix.

Exact fixed commit

uv run --no-dev \
  --with "pytest-asyncio-cooperative @ git+https://github.com/willemt/pytest-asyncio-cooperative@3cac81899122a5034405feaf38ec078e39100ddf" \
  pytest -p no:asyncio \
  examples/pr86/test_workload.py \
  --max-asyncio-tasks=4 \
  --poolwatch \
  --poolwatch-underfill-threshold=0.05 \
  --poolwatch-json=.poolwatch/pr86-after.json \
  --poolwatch-html=.poolwatch/pr86-after.html

uv run --no-dev python examples/pr86/assert_report.py \
  after .poolwatch/pr86-after.json

The checker requires:

• peak concurrency exactly 4 in both runs;
• material underfill before the fix;
• no material underfill after the fix.

A verified Windows/Python 3.13.5 run measured 34.2% utilization and 0.885s underfill before the fix, versus 73.1% utilization with no material underfill at the fixed commit. Those values are observations, not assertions.

Cloud-job simulation

This workload creates 800 async tests by default. Each test:

1. loads a small JSON config;
2. submits it to an in-memory service;
3. polls deterministic simulated remote work;
4. retrieves and validates the result and config checksum.

One virtual minute is compressed to 10 ms. Jobs represent 1–5 virtual minutes, so the complete suite produces realistic completion pressure without sleeping for real minutes.

uv run --no-dev \
  --with "pytest-asyncio-cooperative @ git+https://github.com/willemt/pytest-asyncio-cooperative@3cac81899122a5034405feaf38ec078e39100ddf" \
  pytest -p no:asyncio \
  -W "ignore:FixtureDef.has_location is deprecated:pytest.PytestRemovedIn10Warning" \
  examples/cloud_jobs/test_cloud_jobs.py \
  --max-asyncio-tasks=40 \
  --poolwatch \
  --poolwatch-json=.poolwatch/cloud-jobs.json \
  --poolwatch-html=.poolwatch/cloud-jobs.html

uv run --no-dev python examples/cloud_jobs/assert_report.py \
  .poolwatch/cloud-jobs.json

The checker verifies:

• 800 collected and observed attempts;
• unique node IDs;
• all attempts passed and are complete;
• configured and peak concurrency 40;
• original pytest exit status zero.

A verified Windows/Python 3.13.5 run completed in 2.13s, reached 40/40 peak concurrency, measured 89.0% utilization, and found no scheduler underfill.

Change workload size

pytest -p no:asyncio \
  examples/cloud_jobs/test_cloud_jobs.py \
  --cloud-job-count=3000 \
  --max-asyncio-tasks=150 \
  --poolwatch \
  --poolwatch-json=.poolwatch/cloud-jobs-3000.json

Validate matching expectations:

python examples/cloud_jobs/assert_report.py \
  .poolwatch/cloud-jobs-3000.json \
  --expected-tests=3000 \
  --expected-concurrency=150

Important compatibility note

The cooperative examples disable pytest-asyncio with -p no:asyncio because the workload is scheduled by pytest-asyncio-cooperative.

They do not use xdist. pytest-xdist and pytest-asyncio-cooperative have overlapping runtest-loop implementations and are not a supported combination. See Compatibility.