Skip to content

Releases: breadMSA/pytest-tia

v1.1.1 — Trusted Publishing pipeline

20 Jun 05:43

Choose a tag to compare

Operational release — no functional change to tia itself.

  • Adds PyPI Trusted Publishing (OIDC): releases now build and publish automatically on a version-tag push, with no stored API token. A guard refuses to publish if the tag doesn't match the packaged version, and twine check validates metadata first.
  • This version was the first cut through that pipeline, end to end.

pip install pytest-tia

v1.1.0 — type-only filter, native S3/GCS, GitHub Step Summary

20 Jun 05:43

Choose a tag to compare

Theme: make tia painless to adopt in a real CI pipeline — without trading away the no-false-negative guarantee.

✨ New features

  • Type-only change filter. Edits that are provably dead at runtime no longer trigger tests: if TYPE_CHECKING: blocks and function-local annotations are ignored. Crucially, function-signature and class/dataclass-field annotation changes are kept as semantic — because dataclasses / pydantic / attrs read __annotations__, and silently skipping those would be a false negative. So "added the type hints" stops running the suite, without ever hiding a real dataclass field-type change.
  • Native S3 / GCS map stores. A remote can now be s3://bucket/prefix or gs://bucket/prefix, alongside a plain directory or the bundled HTTP store. The cloud SDKs are imported lazily (pip install pytest-tia[s3] / pytest-tia[gcs]), so the core install stays dependency-light.
  • GitHub Step Summary. tia run auto-detects $GITHUB_STEP_SUMMARY (or --report PATH / --report -) and posts a Markdown impact table to the PR — which files changed, the impact of each, and exactly which tests were selected and why.

📊 Honesty & benchmarks

  • New modular-repo benchmark on boltons: ~96% median skip on real logic changes, versus Flask's ~21% worst case. The README now leads with the honest range, ~21% ↔ ~96%, both measured by replaying real commits on real third-party suites — with the "this looked too good, here's why it's real" audit included.
  • New adversarial mutation test: it mutates every covered function and asserts tia re-selects every test that exercises it. Verified to fail when a false negative is deliberately injected, so it can't pass vacuously.

🐛 Fixes

  • Markdown report no longer crashes on a legacy-codepage console (a real cp950 UnicodeEncodeError when emitting emoji) — it now writes UTF-8 through stdout.buffer.

pip install pytest-tia

tia v1.0.0 — Test Impact Analysis for pytest

19 Jun 12:46

Choose a tag to compare

tia runs only the tests your change actually affects. Record a per-test
map once; then a git diff selects the minimal set to run.

Read this first: honest disclosure

While benchmarking on Flask, an early run reported a 73% median skip
rate
. It looked great. It was a lie — coverage.py's default core on
Python 3.12+ (sysmon) records only the first test to hit each line and
silently drops the rest, so any test reusing a shared helper went unmapped
and would have been skipped. We forced the C tracer, every test mapped,
and the number fell to the truth: ~21% median skip on Flask (≈40%
mean; cosmetic commits 99%+). Flask is small and tightly coupled — close
to the worst case — and we publish that floor on purpose rather than a
cherry-picked ceiling. Full per-commit table: benchmark/RESULTS.md.

The cardinal sin of test selection is skipping a test that would have
failed. Every design choice here is bent toward never doing that.

What's in 1.0

  • Method-level analysis — maps changes to functions, immune to line shifts.
  • Silent dependencies — tracks non-.py files each test opens (config,
    fixtures, templates) via an open audit hook, so a config.yaml change
    selects exactly its readers — not a silent miss.
  • CI mode — maps bake in their own line→function tables (shallow-clone
    safe) and publish to a shared store; tia serve is a zero-dependency
    HTTP backend. GitHub Actions template included.
  • Dynamic-dispatch safety net — detects reflection (getattr by name,
    eval, importlib, __getattr__) and degrades to file-level there.
  • Cosmetic-change filter — ignores comment/docstring/format-only edits
    via AST comparison, so "fix typo" / "run black" selects nothing.

Honest limitations

Subprocess/other-thread code isn't traced; cross-file dynamic dispatch is
beyond it (undecidable in general — mitigation, not a solution). Run the
full suite on a cadence as a safety net. See the README for the full list.