PDFCLOUD-5595 Improve testing speed

[datalogics-kam]

PDFCLOUD-5595

Why this change

Our CI runtime and flake exposure were higher than needed because live API tests ran in the full tests matrix on both pushes and pull requests across Python 3.10–3.14. That made every update slower and tied routine feedback to external service availability. This change keeps live coverage as a quality gate while reducing redundant cost on everyday branch activity.

After splitting live/non-live lanes, we also needed to preserve function-level coverage for customer-facing classes without relying on live suites, because the class-function coverage gate is evaluated from the non-live coverage artifact.

What changed (high level)

We split test execution into non-live and live lanes, then scoped where each lane runs:

• Added a live pytest marker in pyproject.toml.
• Added automatic live-test tagging in tests/conftest.py for:
  • tests under tests/live/
  • tests named test_live_*
• Updated test-and-publish.yml:
  • tests job now runs only non-live tests (-m "not live") across Python 3.10–3.14.
  • Introduced a dedicated live-tests job running only live tests (-m live) on Python 3.11.
  • live-tests runs on pull_request and nightly schedule (cron).
  • Increased CI pytest worker count to -n 5 for both non-live and live jobs.
• Moved class-based live file download tests from tests/test_files.py to tests/live/test_live_file_downloads.py so they are unambiguously routed to the live lane.

Coverage follow-up (non-live gate hardening):

• Added/updated endpoint-owned sync+async unit tests in each endpoint’s home module to explicitly cover optional payload branches (output, output_prefix, pages, page_groups, redaction preview payloads) that were previously under-covered when live tests moved out of the non-live lane.
• Added async preview_redactions unit coverage in tests/test_pdf_redaction_preview.py.
• Updated TESTING_GUIDELINES.md to require endpoint-home test ownership and optional-branch coverage in both sync and async customization/success tests.

Tradeoff: live coverage is no longer exercised in all Python versions on every push, but remains enforced before merge and is checked nightly.

Behavior changes

CI behavior now differs by event type:

• push:
  • tests matrix (3.10–3.14): non-live only, 5 workers.
  • examples matrix unchanged.
• pull_request:
  • same non-live matrix as push.
  • plus live-tests gate on Python 3.11, 5 workers.
• nightly schedule:
  • runs live-tests on Python 3.11.
• release:
  • publish still depends on docs/tests/examples; live tests are no longer in the release dependency path.

Test location behavior:

• TestLiveFileDownloads and TestLiveAsyncFileDownloads now execute only from tests/live/test_live_file_downloads.py.
• Mocked download coverage remains in tests/test_files.py (TestDownloadHelpers and TestAsyncDownloadHelpers) for non-live gating.
• Endpoint branch coverage is now asserted in each endpoint’s own test file rather than a cross-endpoint helper module.

No SDK runtime/API behavior changed; this is CI/test routing and test organization only.

Validation

Validated the live/non-live split and test routing:

• uv run pytest --collect-only -m live -q
• uv run pytest --collect-only -m "not live" -q
• uv run pytest --collect-only -m "not live" -q tests/test_files.py tests/live/test_live_file_downloads.py
• uv run pytest --collect-only -m live -q tests/test_files.py tests/live/test_live_file_downloads.py

Validated endpoint-home coverage updates:

• uv run ruff check on modified endpoint test modules
• uv run pytest on modified endpoint test modules

Validated non-live coverage gate behavior with the same failing command path:

• uvx nox --python 3.10 --session tests -- -n 5 -m "not live"
• uv run python scripts/check_class_function_coverage.py coverage/py3.10/coverage.json --class PdfRestClient --class AsyncPdfRestClient --class _FilesClient --class _AsyncFilesClient --fail-under 90 --markdown-report coverage/py3.10/class-function-coverage.md

Result: class-function coverage gate passes from non-live coverage artifacts.

Limitation: full multi-version nox matrix was not rerun locally for this follow-up; validation focused on the exact non-live gate path and impacted test modules.

Risks and follow-ups

Main risk is classification drift: a new live test that does not live under tests/live/ and does not use the test_live_* naming pattern could be routed incorrectly. The current hook mitigates known patterns, and the moved class-based download tests remove one concrete misclassification source.

Follow-ups to consider:

• Optionally add a lint/check that enforces live-test naming/location conventions.
• Monitor first few nightly runs for duration/flakiness and tune worker count if needed.
• If desired, add a lightweight report that surfaces live lane pass rate trends over time.

[netlify]

✅ Deploy Preview for pdfrest-python ready!

Name	Link
🔨 Latest commit	a60ea19
🔍 Latest deploy log	https://app.netlify.com/projects/pdfrest-python/deploys/6998b6aed6295f00075f0a40
😎 Deploy Preview	https://deploy-preview-31--pdfrest-python.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 40211008af

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[datalogics-cgreen]

LGTM

No critical findings from Codex audit