Repository navigation
Releases
TestDelta v0.1.0a3 — Existing PRs and isolated GitHub Action
Existing test-only pull requests can now be compared with an external manifest, without rewriting their commits. The native GitHub Action runs TestDelta in its own environment and preserves the caller’s project Python and dependencies.
- Exact external-config bytes/hash are recorded and checked by replay; the original config file can be removed after comparison.
- Evidence includes bounded changed-test and actual mutant diffs, full-diff hashes, per-run UTC timestamps, and a conservative suite-call estimate.
testdelta exportand automatic job summaries omit raw logs, source/config text, diffs, and absolute runtime paths. Full local reports remain available.- A selected-interpreter regression was fixed: the collector is now loaded alone rather than exposing the supervisor’s site-packages to project tests.
Validation: 194 source tests and lint passed locally. The frozen 30-development/20-validation corpus matched all 50 declared outcomes: 17 loss,17 no-loss,16 invalid-baseline inconclusive. These hand-authored controls test classification behavior separately from automatic mutation quality. GitHub CI verifies the four supported Python/OS combinations, full corpus, Ubuntu/macOS native Action, and installed-wheel comparison/replay.
A fixed local pricing benchmark completed 20/20 repetitions with 10 Cosmic Ray mutants each, no exclusions/errors/retries, p95 27.697 seconds and maximum 27.963 seconds. Setup exclusions, per-run evidence, uncontrolled host load and exact scope are recorded in docs/performance.md. This is not an arbitrary-project or cross-platform performance bound.
This remains an early alpha, distributed here rather than assumed on PyPI. Trusted pytest code retains host filesystem/network permissions; no network-blocked execution profile or full dependency/OS replay image is provided. Mixed source/test changes, Windows and plugin/environment-dependent suites remain outside the supported initial scope. The two recorded Boltons compatibility checks remain inconclusive. No general accuracy, algorithmic novelty, adoption or star-growth claim is made. See docs/quality-status.md for deferred original-plan targets.
Assets contain the wheel, source distribution, and SHA256SUMS. Raw local evidence and virtual environments are excluded.
v0.1.0a2 — Readable evidence and real repository checks
TestDelta a2 makes the execution evidence easier to review and fixes import-origin handling for additional Python layouts.
Changes:
- Structured offline HTML and Markdown: verdict, selected scope, mutation outcomes, controls, diagnostics, and source/test fingerprints.
- Unconfirmed findings stay provisional; incomplete records cannot produce a clean presentation or successful comparison exit.
- Import-origin checks cover
lib/and package layouts while avoiding unrelated standard-library basename conflicts. - Target-line tracing caches path resolution. A real Boltons case that previously exceeded the unchanged 15-second baseline limit now completes its old/new baselines in under a second in the recorded local runs.
- Pinned real-repository reproduction script, sanitized evidence for two Boltons test-only PRs, and an optional coding-agent review skill.
- Controlled handling of deleted production source in unsupported plans.
Validation: 130 local tests and Ruff passed; installed-wheel lost/refactor/inconclusive demos passed. Commit CI covers Ubuntu Python 3.11–3.13 and macOS Python 3.13, including wheel installation and demo smoke.
Both real-repository comparisons remain inconclusive: one has unresolved runtime-error pairs, the other an old collection/new runtime-error pair. Attempting every selected mutant is not a complete measurement. The evidence is not proof of upstream bugs, maintainer adoption, or general test-quality accuracy. See compatibility details.
The same-mutation technique is established work. This early alpha supports small deterministic pytest test-only changes and trusted local execution. No PyPI publication or measured agent-repair success rate is claimed.
The wheel, source archive, and SHA-256 checksums are attached.
v0.1.0a1 — Initial pytest comparison alpha
TestDelta's first executable alpha reviews pytest test-only changes using identical Cosmic Ray mutations on unchanged production code.
Included:
init,plan,compare,replay, and three executed synthetic demo scenarios.- Committed Git snapshots, explicit manifests, source fingerprints, import-origin and line-reach evidence.
- Repeated lost-fault confirmation and final controls; errors and incomplete work never become clean results.
- Fresh bytecode-cache namespaces, process timeouts, and a bounded mutation-provider worker.
- Local JSON/Markdown/HTML reports, replay bundles, and GitHub Actions job summaries.
- English/Korean documentation, MIT license, and a solo-maintainer workflow.
Validation for commit 3cf037169bbb05431d769cd619ff656061de3302: 61 tests; lint; wheel build; installed-wheel smoke across Ubuntu Python 3.11–3.13 and macOS Python 3.13. CI evidence.
This is an early alpha for small deterministic pytest projects. Run trusted local tests only: temporary workspaces are not a sandbox. Source/configuration/dependency changes, external services, Windows, and arbitrary monorepos are outside this release's supported scope. A finite mutation result is review evidence, not proof of software correctness. The same-mutation method is established work; see the README's alternatives.
The wheel, source archive, and SHA-256 checksums are attached. This is a GitHub prerelease; no PyPI publication is claimed.
TestDelta Test Quality v0.1.0-alpha.4
TestDelta compares old and new pytest tests against unchanged production code and the same selected faults, then supplies reviewable evidence when a changed test loses detection.
This alpha improves the Action's setup isolation: caller Python modules no longer shadow bootstrap or mutation-worker imports, and caller pip configuration cannot redirect installation out of the separate tool environment. The caller's prepared project Python and dependencies remain separate. The Marketplace display name is TestDelta Test Quality.
The release also includes an Action-first English/Korean README, an interactive static illustration backed by an executed synthetic case, an actual evidence page, a 1200×630 share image, and ready-to-copy announcement drafts. The website runs no tests or model calls and includes no trackers or third-party fonts.
Validation for commit 55d64056c14834eab99cf49ca86c4786a14804bd: all declared CI jobs passed, covering 195 source tests per supported matrix, all 50 frozen conformance cases, Ubuntu/macOS native Action traps, clean installed-wheel comparison/replay, and Pages deployment. The 50 hand-authored cases do not establish automatic mutation quality or general accuracy. The recorded a3 runtime measurement and both inconclusive Boltons compatibility checks retain their original scope.
Supported initial scope: committed test-only pytest changes, explicit Python source/test files, Python 3.11–3.13, Ubuntu/macOS. Production source, helpers, configuration and dependencies must remain unchanged. Trusted test code retains host filesystem/network permissions; temporary workspaces are not a sandbox. This is not a stable release, a PyPI publication, or a guarantee about AI-generated tests.
Assets: Python wheel, source distribution, SHA256SUMS-alpha4, and the public press kit. Private dataset archives, raw local evidence and virtual environments are excluded.