v0.8.0 — Per-test dynamic evidence

v0.8.0 — Per-test dynamic evidence

PR Test Guard can now connect changed Python lines to the pytest tests that
actually executed them when context-aware coverage data is supplied.

Highlights

• Added normalized per-test coverage-context ingestion for coverage.py JSON.
• Added runtime execution evidence, exact parameterized test IDs, and
  setup/run/teardown phases to changed-symbol reports.
• Added advisory PTG010 for deterministically related tests that were
  observed at runtime but did not execute the changed behavior.
• Kept existing XML coverage and PTG001–PTG009 behavior backward compatible
  when --coverage-contexts is omitted.
• Added CLI, structured JSON, text, GitHub Summary, and GitHub Action support
  for context-aware coverage evidence.
• Added paired controlled fixtures for runtime mismatch and runtime overlap.

Generate compatible data with pytest-cov and coverage.py dynamic contexts:

pytest --cov=src --cov-context=test
coverage json --show-contexts -o coverage-contexts.json
pr-test-guard check \
  --base origin/main \
  --coverage coverage.xml \
  --coverage-contexts coverage-contexts.json

This remains advisory evidence: execution does not by itself prove behavioral
correctness or assertion adequacy. PTG010 stays inconclusive when paths cannot
be mapped reliably, related tests do not appear in the artifact, contexts are
unknown, or the artifact is incomplete.

Known limits

• Python/pytest context IDs only.
• Context-aware coverage input is opt-in; PR Test Guard does not instrument the
  target repository automatically.
• No branch-level per-test mapping yet.
• No business-intent correctness inference.

PR Test Guard v0.7.0

PR Test Guard v0.7.0

Version 0.7.0 connects changed Python behavior to direct test and assertion
evidence, then adds an opt-in check that restores compatible base behavior.

What Changed

• Added a change-evidence record for every changed Python function or method,
  including behavior shapes, related tests, direct-call evidence, direct mock
  replacement, and assertion-flow status.
• Added PTG007 when a changed function or method has no deterministic direct
  or bounded static call-path test evidence. Evidence propagates through
  resolvable calls between changed functions; direct mock replacements stay
  under PTG005 to avoid duplicate findings.
• Added bounded intraprocedural result flow through local assignments,
  attributes, and subscripts.
• Added PTG008 when a real direct call to changed behavior is unobserved or
  reaches only a weak assertion. Existence checks and self-comparisons do not
  count as meaningful constraints.
• Added function-level base-behavior rollback probes to --deep. Candidates
  require matching base/PR signatures and direct-call test evidence.
• Added PTG009 when the PR baseline passes and restoring the base function
  implementation also passes. Collection, import, and syntax failures are
  reported as inconclusive.
• Added change_units, assertion_flows, and rollback_probes to JSON output.
• Extended rule policy configuration to PTG007, PTG008, and PTG009.

Compatibility

The CLI and Action inputs are unchanged. The three new rules are advisory by
default, so existing consumers may observe additional findings and higher
finding counts. Repositories can set any new rule to off, warn, or error
using the existing rule policy.

Rollback probes use the existing --deep, --test-command, and --max-probes
options. They run in a separate isolated Git worktree and never modify the
checked-out repository.

Validation

.venv/bin/python -m pytest -q
.venv/bin/python -m pr_test_guard validate-cases --run
.venv/bin/python scripts/validate_real_pr_bundles.py
.venv/bin/python -m pr_test_guard validate-config --format json
.venv/bin/python -m pr_test_guard --version
python3 -m pip wheel --no-build-isolation --no-deps --wheel-dir /tmp/pr-test-guard-v0.7.0-dist .
git diff --check

PR Test Guard v0.6.0

PR Test Guard v0.6.0

Version 0.6.0 makes GitHub Actions adoption quieter, more predictable, and
easier to compose with downstream workflow steps.

What Changed

• Added stable PR Test Guard / PTGxxx titles to warning and error annotations.
• Added independent workflow-command escaping for annotation properties and
  messages, including safe file paths containing %, ,, or :.
• Added github.annotations.max_total and
  github.annotations.max_per_rule configuration.
• Bounded annotations to 50 total and 10 per rule by default, with error-level
  findings selected before warnings and deterministic rule/path/line ordering.
• Reported total and per-rule annotation omissions in the GitHub job summary.
• Kept GitHub Summary tables and JSON reports complete when annotation limits
  omit workflow commands.
• Added composite Action outputs for status, findings-count,
  warning-count, error-count, and report-path.
• Added explicit operational-error status output when Action validation or the
  checker returns an operational failure.
• Added YAML, JSON, and TOML configuration coverage plus focused annotation,
  escaping, prioritization, zero-limit, summary, JSON, and output-contract tests.

Configuration

github:
  annotations:
    max_total: 50
    max_per_rule: 10

Both values are non-negative integers. 0 disables the corresponding
annotation scope. Limits affect only GitHub workflow-command annotations; they
do not suppress findings, change policy exit codes, truncate summary tables, or
remove findings from JSON output.

Action Outputs

- uses: tigerless-labs/pr-test-guard@v0.6.0
  id: guard
  with:
    base: origin/${{ github.base_ref }}

- if: always() && steps.guard.outputs.status == 'advisory'
  env:
    FINDINGS: ${{ steps.guard.outputs.findings-count }}
  run: echo "$FINDINGS finding(s) need review"

status is clean, advisory, failed, or operational-error. The count
outputs reflect the final policy-filtered result.

Compatibility

• Detection rules, finding schemas, policy exit codes, and existing Action
  inputs are unchanged.
• Valid 0.5.1 configuration remains valid; normalized policy JSON now includes
  the effective GitHub annotation limits.
• Default GitHub annotation delivery is now bounded. Full findings remain
  available in the job summary and JSON report.
• Text and JSON CLI output behavior is unchanged.

Validation

.venv/bin/python -m pytest -q
.venv/bin/python -m pr_test_guard validate-cases --run
.venv/bin/python scripts/run_case.py --output-dir /tmp/pr-test-guard-v0.6.0-artifacts
.venv/bin/python scripts/validate_real_pr_bundles.py
.venv/bin/python -m pr_test_guard validate-config --format json
.venv/bin/python -m pr_test_guard --version
python3 -m pip wheel --no-build-isolation --no-deps --wheel-dir /tmp/pr-test-guard-v0.6.0-dist .
git diff --check

PR Test Guard v0.5.1

PR Test Guard v0.5.1

Version 0.5.1 makes repository configuration safer to adopt and easier to
verify before a pull-request analysis runs.

What Changed

• Added pr-test-guard validate-config for configuration preflight without a
  Git repository, base ref, or PR diff.
• Added --format json to print the normalized effective configuration for CI
  checks and other automation.
• Reject unknown fields at every supported configuration level, including
  source-to-test mapping entries.
• Suggest nearby supported field names in typo errors, such as
  max_candidate -> max_candidates.
• Reject ambiguous rule objects that contain both level and action, or
  neither field.
• Reject rule ids that collide after case normalization.
• Reject empty path patterns and booleans supplied for
  related_tests.max_candidates.
• Apply repository-relative path validation consistently to paths.ignore,
  paths.tests, and related_tests.mappings.
• Scope default pytest discovery to the product test suite so pytest does not
  collect the nested executable fixture repositories.

Configuration Validation

Validate an automatically discovered configuration:

pr-test-guard validate-config

Validate a specific file and consume the normalized result as JSON:

pr-test-guard validate-config \
  --config path/to/.pr-test-guard.yml \
  --format json

The command exits 0 for a valid file or valid built-in defaults and exits 2
for missing, malformed, or semantically invalid configuration.

Compatibility

• Existing valid YAML, JSON, and TOML configurations retain their behavior.
• Direct boolean rule actions remain supported (true means warn, false
  means off).
• Configuration that previously depended on ignored unknown fields, empty path
  entries, or ambiguous rule objects now fails explicitly instead of silently
  falling back to another behavior.
• Detection rules, finding schemas, policy exit codes, and GitHub Action inputs
  are unchanged.

Validation

.venv/bin/python -m pytest -q
.venv/bin/python -m pr_test_guard validate-cases --run
.venv/bin/python scripts/run_case.py --output-dir /tmp/pr-test-guard-v0.5.1-artifacts
.venv/bin/python scripts/validate_real_pr_bundles.py
.venv/bin/python -m pr_test_guard validate-config --format json
.venv/bin/python -m pr_test_guard --version
python3 -m pip wheel --no-build-isolation --no-deps --wheel-dir /tmp/pr-test-guard-v0.5.1-dist .
git diff --check

PR Test Guard v0.5.0

PR Test Guard v0.5.0

Version 0.5.0 adds explicit source-to-test path mappings for stable test
relationships that cannot be inferred from Python imports, direct calls, test
names, or mock targets.

What Changed

• Added related_tests.mappings configuration with one repository-relative
  source glob and one or more tests globs per mapping.
• Merged configured candidates with existing deterministic related-test
  discovery instead of replacing inferred relationships.
• Added the inspectable configured_path_mapping reason and matched changed
  source paths to related-test output.
• Kept overlapping configured and inferred candidates deduplicated while
  retaining all applicable reasons and changed symbols.
• Supported path mappings for module-level changes even when no changed Python
  class or function symbol can be collected.
• Applied the existing test classifier before configured mappings, so
  paths.tests.exclude continues to take precedence.
• Added YAML, JSON, and TOML coverage plus validation for malformed, absolute,
  parent-relative, and empty mapping definitions.
• Documented that mappings provide review context rather than evidence that a
  test ran or sufficiently validates the change.

Configuration

related_tests:
  max_candidates: 5
  mappings:
    - source: "src/payments/**"
      tests:
        - "tests/integration/payments/**"

When a changed production path matches source, every test_* function in a
matched tracked Python test file becomes a related-test candidate. tests may
also be a single string. Multiple mappings and multiple matching source paths
are combined deterministically.

Mappings do not suppress PTG001 when the pull request has no test-file change,
do not claim that a candidate test executed, and do not change merge policy.

Compatibility

• Repositories without related_tests.mappings retain the previous candidate
  discovery behavior.
• Existing rule ids, severity policy, exit codes, CLI options, and GitHub Action
  inputs are unchanged.
• Related-test JSON objects now include matched_sources; it is an empty list
  for candidates found only through existing symbol relationships.
• Effective policy JSON now includes related_tests.mappings, which is an empty
  list when no mappings are configured.
• Human and GitHub summaries label the related-test evidence column as changed
  targets so it can represent either symbols or configured source paths.

Validation

.venv/bin/python -m pytest -q tests
.venv/bin/python -m pr_test_guard validate-cases --run
.venv/bin/python scripts/run_case.py --output-dir /tmp/pr-test-guard-v0.5.0-artifacts
.venv/bin/python scripts/validate_real_pr_bundles.py
.venv/bin/python -m pr_test_guard --version
python3 -m pip wheel --no-build-isolation --no-deps --wheel-dir /tmp/pr-test-guard-v0.5.0-dist .
git diff --check

PR Test Guard v0.4.0

PR Test Guard v0.4.0

Version 0.4.0 adds repository-configurable test path recognition so the
checker can handle Python projects whose tests do not follow the built-in
tests/, test_*.py, or *_test.py conventions.

What Changed

• Added paths.tests.include globs to recognize repository-specific test
  layouts such as spec/ and *_spec.py.
• Added paths.tests.exclude globs to remove false test classifications;
  excludes take precedence over both configured includes and built-in matches.
• Applied one shared test-path classifier to PTG001, PTG003, PTG004, PTG005,
  related-test discovery, changed-code selection, and text/JSON report counts.
• Removed the reporter's duplicate test-path heuristic so analyzer and output
  classification cannot drift independently.
• Added repository-relative pattern validation, slash normalization, and JSON
  policy output for the effective include/exclude configuration.
• Documented the difference between test classification and paths.ignore,
  which remains a post-detection finding filter.

Configuration

paths:
  tests:
    include:
      - "spec/**"
      - "**/*_spec.py"
    exclude:
      - "src/test_support/**"

Configured includes extend the built-in conventions. Excludes win when a path
matches both lists or when a built-in test path should be treated as production
code. Patterns are matched against normalized repository-relative paths.

Compatibility

• Repositories without paths.tests configuration retain the previous test
  path behavior.
• Rule actions, policy.fail_on, paths.ignore, and exit-code behavior are
  unchanged.
• JSON policy output adds paths.tests.include and paths.tests.exclude
  fields; existing fields remain compatible.
• The GitHub Action needs no new input because its existing config input
  already passes repository configuration to the CLI.

Validation

.venv/bin/python -m pytest -q tests
.venv/bin/python -m pr_test_guard validate-cases --run
.venv/bin/python scripts/run_case.py --output-dir /tmp/pr-test-guard-v0.4.0-artifacts
.venv/bin/python scripts/validate_real_pr_bundles.py
.venv/bin/python -m pr_test_guard --version
python3 -m pip wheel --no-build-isolation --no-deps --wheel-dir /tmp/pr-test-guard-v0.4.0-dist .
git diff --check

PR Test Guard v0.3.0

PR Test Guard v0.3.0

Version 0.3.0 is a minor release focused on making real-PR dogfooding easier to run and summarize from checker JSON reports.

What Changed

• Added scripts/draft_dogfood_review.py to turn pr-test-guard check --json-output reports into local raw dogfood review drafts.
• Preserved local-only review context in drafts, including file, line, severity, message, and evidence, so reviewers can label findings before sanitizing them.
• Kept scripts/sanitize_dogfood_review.py as the public-safe conversion step for removing repository-specific details before records are shared or committed.
• Expanded dogfood summaries with per-rule label rates, top evidence shapes, and action counts.
• Documented the JSON report -> raw draft -> sanitized record -> aggregate summary workflow in the dogfooding guide.

Compatibility

• Default checker behavior, CLI options, and GitHub Action inputs remain compatible with 0.2.5.
• The new dogfooding draft helper is development workflow tooling and does not affect normal pr-test-guard check output.
• Sanitized public records continue to use schema version 1.

Validation

.venv/bin/python -m pytest tests
.venv/bin/python -m pr_test_guard validate-cases --run
git diff --check

PR Test Guard v0.2.3

PR Test Guard v0.2.3

Version 0.2.3 is a patch release focused on reviewer context and evidence clarity for the direct PR checker.

What Changed

• Added deterministic related-test candidates for changed Python symbols using imports, direct calls, supported mock targets, and guarded test-name context.
• Included related-test context in JSON output and summarized candidate counts in text and GitHub output.
• Added related-test context to PTG001, PTG003, and PTG006 evidence where it helps explain nearby test signals.
• Improved PTG005 relationship evidence with stable key/value context and clearer constrained dependency-mock suppression notes.
• Added public-safe dogfood-derived PTG005 controls, including an unconstrained helper-mock fixture and sanitized review example.

Compatibility

• Advisory behavior remains unchanged; findings still do not fail a job by default.
• The default checker still requires no LLM, API key, hosted service, or GitHub write permission.
• Existing CLI and GitHub Action inputs remain compatible.
• Related-test candidates are review context only. They do not prove that a test validates the changed behavior.

Validation

.venv/bin/python -m pytest tests
.venv/bin/python -m pr_test_guard validate-cases --run
git diff --check

PR Test Guard v0.2.2

PR Test Guard v0.2.2

Version 0.2.2 is a patch release focused on PTG006 probe-rerun correctness and validation coverage.

What Changed

• Fixed a PTG006 correctness issue where Python could reuse stale .pyc bytecode after a targeted probe rewrote a source line to the same byte length.
• PTG006 now clears the target file's bytecode cache after applying and restoring a probe in the isolated worktree.
• Added validation controls for:
  • strong tests that kill status-code probes;
  • baseline test-command failure skipping PTG006;
  • max_probes limiting generated/applied probes;
  • if-condition comparison probes;
  • chained comparison probes;
  • unsupported response-constructor and symbolic HTTP status shapes.

Why It Matters

PTG006 is useful only when a surviving probe reflects actual test insensitivity.
When Python reused stale bytecode, a probe could appear to survive even though
the changed source should have failed the configured tests. This release removes
that false-survivor path while keeping deep probes bounded and opt-in.

Compatibility

The public CLI and Action inputs are unchanged. --deep remains opt-in and still
requires an explicit test command.

Validation

.venv/bin/python -m pytest tests
.venv/bin/python -m pr_test_guard validate-cases --run

PR Test Guard v0.2.1

PR Test Guard v0.2.1

Version 0.2.1 is a patch release focused on PTG005 false-positive reduction.

What Changed

• PTG005 still warns when a mock directly replaces a Python symbol changed by the PR.
• Explicit external-boundary dependency mocks remain suppressed from PTG005 warnings.
• Changed tests that mock direct internal dependencies called on changed production lines are now suppressed when the test constrains the changed owner through:
  • mock interaction assertions such as assert_called_once_with or assert_called_with;
  • call_args or call_count assertions;
  • non-weak assertions over the changed owner result;
  • pytest.raises around the changed owner call.
• Added legitimate_helper_mock_001 as a PTG005 negative-control fixture.

Why It Matters

Mocking a dependency is not automatically weak test evidence. Some tests use a
mock as an interaction boundary while still asserting the behavior changed by the
PR. PTG005 now keeps direct changed-symbol mocks visible, but reduces warnings
for dependency mocks that are constrained by inspectable test assertions.

Behavior Examples

Still warns:

@patch("service.charge")
def test_charge(mock_charge):
    ...

Suppressed as constrained dependency evidence:

@patch("service.send_payload")
def test_process(mock_send):
    process_payload({"id": "42"})
    mock_send.assert_called_once_with({"id": "42", "source": "api"})

Still warns when unconstrained:

@patch("service.send_payload")
def test_process(mock_send):
    mock_send.return_value = True
    assert process_payload({"id": "42"}) is not None

Validation

.venv/bin/python -m pytest tests
.venv/bin/python -m pr_test_guard validate-cases --run

Compatibility

The default path remains deterministic and offline. --deep behavior is
unchanged, and findings remain advisory by default.