tia v1.0.0 — Test Impact Analysis for pytest
tia runs only the tests your change actually affects. Record a per-test
map once; then a git diff selects the minimal set to run.
Read this first: honest disclosure
While benchmarking on Flask, an early run reported a 73% median skip
rate. It looked great. It was a lie — coverage.py's default core on
Python 3.12+ (sysmon) records only the first test to hit each line and
silently drops the rest, so any test reusing a shared helper went unmapped
and would have been skipped. We forced the C tracer, every test mapped,
and the number fell to the truth: ~21% median skip on Flask (≈40%
mean; cosmetic commits 99%+). Flask is small and tightly coupled — close
to the worst case — and we publish that floor on purpose rather than a
cherry-picked ceiling. Full per-commit table: benchmark/RESULTS.md.
The cardinal sin of test selection is skipping a test that would have
failed. Every design choice here is bent toward never doing that.
What's in 1.0
- Method-level analysis — maps changes to functions, immune to line shifts.
- Silent dependencies — tracks non-
.pyfiles each test opens (config,
fixtures, templates) via anopenaudit hook, so aconfig.yamlchange
selects exactly its readers — not a silent miss. - CI mode — maps bake in their own line→function tables (shallow-clone
safe) and publish to a shared store;tia serveis a zero-dependency
HTTP backend. GitHub Actions template included. - Dynamic-dispatch safety net — detects reflection (
getattrby name,
eval,importlib,__getattr__) and degrades to file-level there. - Cosmetic-change filter — ignores comment/docstring/format-only edits
via AST comparison, so "fix typo" / "run black" selects nothing.
Honest limitations
Subprocess/other-thread code isn't traced; cross-file dynamic dispatch is
beyond it (undecidable in general — mitigation, not a solution). Run the
full suite on a cadence as a safety net. See the README for the full list.