Releases: breadMSA/pytest-tia
v1.1.1 — Trusted Publishing pipeline
Operational release — no functional change to tia itself.
- Adds PyPI Trusted Publishing (OIDC): releases now build and publish automatically on a version-tag push, with no stored API token. A guard refuses to publish if the tag doesn't match the packaged version, and
twine checkvalidates metadata first. - This version was the first cut through that pipeline, end to end.
pip install pytest-tia
v1.1.0 — type-only filter, native S3/GCS, GitHub Step Summary
Theme: make tia painless to adopt in a real CI pipeline — without trading away the no-false-negative guarantee.
✨ New features
- Type-only change filter. Edits that are provably dead at runtime no longer trigger tests:
if TYPE_CHECKING:blocks and function-local annotations are ignored. Crucially, function-signature and class/dataclass-field annotation changes are kept as semantic — becausedataclasses/pydantic/attrsread__annotations__, and silently skipping those would be a false negative. So "added the type hints" stops running the suite, without ever hiding a real dataclass field-type change. - Native S3 / GCS map stores. A remote can now be
s3://bucket/prefixorgs://bucket/prefix, alongside a plain directory or the bundled HTTP store. The cloud SDKs are imported lazily (pip install pytest-tia[s3]/pytest-tia[gcs]), so the core install stays dependency-light. - GitHub Step Summary.
tia runauto-detects$GITHUB_STEP_SUMMARY(or--report PATH/--report -) and posts a Markdown impact table to the PR — which files changed, the impact of each, and exactly which tests were selected and why.
📊 Honesty & benchmarks
- New modular-repo benchmark on boltons: ~96% median skip on real logic changes, versus Flask's ~21% worst case. The README now leads with the honest range, ~21% ↔ ~96%, both measured by replaying real commits on real third-party suites — with the "this looked too good, here's why it's real" audit included.
- New adversarial mutation test: it mutates every covered function and asserts tia re-selects every test that exercises it. Verified to fail when a false negative is deliberately injected, so it can't pass vacuously.
🐛 Fixes
- Markdown report no longer crashes on a legacy-codepage console (a real
cp950UnicodeEncodeErrorwhen emitting emoji) — it now writes UTF-8 throughstdout.buffer.
pip install pytest-tia
tia v1.0.0 — Test Impact Analysis for pytest
tia runs only the tests your change actually affects. Record a per-test
map once; then a git diff selects the minimal set to run.
Read this first: honest disclosure
While benchmarking on Flask, an early run reported a 73% median skip
rate. It looked great. It was a lie — coverage.py's default core on
Python 3.12+ (sysmon) records only the first test to hit each line and
silently drops the rest, so any test reusing a shared helper went unmapped
and would have been skipped. We forced the C tracer, every test mapped,
and the number fell to the truth: ~21% median skip on Flask (≈40%
mean; cosmetic commits 99%+). Flask is small and tightly coupled — close
to the worst case — and we publish that floor on purpose rather than a
cherry-picked ceiling. Full per-commit table: benchmark/RESULTS.md.
The cardinal sin of test selection is skipping a test that would have
failed. Every design choice here is bent toward never doing that.
What's in 1.0
- Method-level analysis — maps changes to functions, immune to line shifts.
- Silent dependencies — tracks non-
.pyfiles each test opens (config,
fixtures, templates) via anopenaudit hook, so aconfig.yamlchange
selects exactly its readers — not a silent miss. - CI mode — maps bake in their own line→function tables (shallow-clone
safe) and publish to a shared store;tia serveis a zero-dependency
HTTP backend. GitHub Actions template included. - Dynamic-dispatch safety net — detects reflection (
getattrby name,
eval,importlib,__getattr__) and degrades to file-level there. - Cosmetic-change filter — ignores comment/docstring/format-only edits
via AST comparison, so "fix typo" / "run black" selects nothing.
Honest limitations
Subprocess/other-thread code isn't traced; cross-file dynamic dispatch is
beyond it (undecidable in general — mitigation, not a solution). Run the
full suite on a cadence as a safety net. See the README for the full list.