Surface filter-promoted unraisable warnings directly (#14263)

[paulzuradzki]

Summary

Closes #14263.

filterwarnings = error::ResourceWarning does not fail tests that leak resources through a reference cycle. collect_unraisable wraps the captured warning in PytestUnraisableExceptionWarning, a class the user has not filtered.

collect_unraisable now checks warnings.filters before wrapping. When an active error filter targets a class that the captured warning is an instance of, collect_unraisable re-raises that warning directly; the resulting exception fails the test.

Visual

Behavior on main: this test passes despite user configuring for ResourceWarning to trigger an error. Pictured test_leak.py triggers ResourceWarning.

After PR: raises error as expected

Scope

collect_unraisable chooses between two branches when it processes a queued unraisable:

• Unwrap (new in this PR): re-raise the unraisable's inner warning class directly, so the user's error::<class> filter fails the test as written. Applies only when an active error::<class> filter matches that class.
• Wrap (pre-PR behavior, still the default): wrap the unraisable in PytestUnraisableExceptionWarning and emit it via warnings.warn. A __del__ that raises a Warning without a matching error::<class> filter still hits this branch.

Suites without error::<warning> filters see no change.

The unwrap-branch filter check is action == "error" plus issubclass(warning_class, filter_category). It does not re-run Python's warnings module's full match logic, which additionally tests the filter's message_regex, module_regex, and lineno fields. Consequence: for users with narrowly-scoped filters like error:some_msg:ResourceWarning::42, this branch can fail a test on a ResourceWarning whose message wouldn't have matched the regex. For broad filters like error::ResourceWarning the behavior matches user intent.

Branch structure

Three red→green commit pairs plus a comment-cleanup commit. Each red commit fails the named test; the next commit makes it pass.

1. 7aea99ab0 (red) → 7cfadea30 (fix): test_refcycle_resource_warning_filter exercises the user-visible wrap-vs-unwrap behavior covered in Summary.
2. 485f342fb (red) → cd7e0432e (fix): the move-GC hardening from Fix session-end gc in unraisableexception plugin to respect warning filters #14273. test_unraisable_decouples_from_cleanup_stack_order uses @hookimpl(trylast=True) to force a LIFO cleanup-stack order that PR apply warnings filter as soon as possible, and remove it as late as possible #13057 incidentally avoided. The fix moves gc.collect() and queue processing into pytest_unconfigure, which runs before _cleanup_stack.close().
3. d22468f2b (red) → fb50e17dc (fix): test_pytest_unconfigure_survives_failed_pytest_configure raises UsageError from a conftest's pytest_configure, which exposed a KeyError after step 2 moved the GC call out of config.add_cleanup. The fix adds a stash-presence check at the top of pytest_unconfigure.

Follow-ups: f5311b7a7 tightens the new comments in unraisableexception.py; 6e8ca5ba8 adds failure messages to the new tests' result.ret assertions.

Rebased on current main: 8f303fc58 reconciles with #14441 (which reduced the default gc_collect_harder passes to 1 on CPython, 5 on PyPy). That tweak lived in cleanup(), which this branch empties, so the commit carries the same default into the new pytest_unconfigure location rather than silently reverting #14441.

Relationship to #14273

#14273 attempted approach 2 from the issue (move GC to pytest_unconfigure). Maintainers closed it unmerged: #13057 (Dec 2024) had already shipped approach 1, so the move-GC change alone produced no observable behavior difference and no fails-then-passes regression test was possible. @Zac-HD asked: "we'd need to see a regression test which demonstrates that the unraisable hook is now subject to warnings."

Step 2 in the branch structure addresses that question. The red test forces the LIFO ordering that #13057 incidentally avoided, demonstrating the structural fragility was load-bearing on plugin registration order rather than benign.

Testing

Reproducer

My adapted version of the original gist from the issue. The original references an undefined stderr_lines in its run_scenario helper; the adapted version applies a one-line fix.

For a one-file check, save this as test_leak.py:

import gc; gc.disable()  # let the cycle survive to session-end gc

def test_it():
    f = open(__file__)
    cycle = [f]
    cycle.append(cycle)

Then run pytest -W error::ResourceWarning test_leak.py. The -W flag drives the same code path as the pytest.ini filterwarnings setting. On main the test passes (bug); on this branch it exits 1 with the inner ResourceWarning raised directly, no PytestUnraisableExceptionWarning wrapper.

If you run from inside the pytest source tree, the repo's pyproject.toml [tool.pytest] filterwarnings = ['error', ...] promotes every warning to an error. Comment out the bare 'error' entry, or run the snippet from outside the repo.

• Regression tests
• Manually verify bug on main
  • Check out main, install (pip install -e .), run the adapted reproducer. Scenario 1 exits 0 (bug confirmed). Scenario 2 exits 1.
• Manually verify branch fix. Check out branch, run the same reproducer. Scenario 1 now exits 1 with the inner ResourceWarning raised. Scenario 2 still exits 1.
• Check out any of the red commits (7aea99ab0, 485f342fb, d22468f2b); the named test fails. Check out the next commit; it passes.

Snippets

# -W error::ResourceWarning flag not needed for these (no-op) since the same setting is config'ed in pytester unit test init file

##########################
# red ❌
git checkout 92c7681a2 && pytest testing/test_unraisableexception.py

# green ✅
git checkout 4cf46d4f9 && pytest testing/test_unraisableexception.py

##########################
# red ❌
git checkout 21d334ad1 && pytest testing/test_unraisableexception.py

# green ✅  
git checkout 839a20a0c && pytest testing/test_unraisableexception.py

##########################
# red ❌
git checkout dead24a90 && pytest testing/test_unraisableexception.py

# green ✅
git checkout ef0d16180 && testing/test_unraisableexception.py

External suite tests

(time-permitting; I don't think this needs to block review)

• Verify the scope guard. Run a suite that raises a Warning from __del__ without a matching error filter against this branch.
• Run a real project that uses error::DeprecationWarning or error::UserWarning filters against this branch. Nothing previously-passing should now fail.

AI disclosure

• Yes. Used AI assist with code writing and learning artifacts.
• Knowledge gaps on my side: new to repo. Spent a good amount of time researching how Python reclaims memory, sys.unraisable hook, and manually running the regression test before and after changes.
• References used
  • https://adamj.eu/tech/2025/01/02/python-unraisiable-exceptions/
  • https://adamj.eu/tech/2025/01/07/python-test-unraisable-exceptions/
  • https://www.youtube.com/watch?v=G4Lya0KVG1E
  • https://vstinner.github.io/sys-unraisablehook-python38.html
  • https://docs.python.org/3.11/library/sys.html#sys.unraisablehook

Side note: open to feedback to pull some of the PR comments into the code if helpful

Changelog

changelog/14263.bugfix.rst.

[FuzzysTodd]

pytest_unconfigure fires before _cleanup_stack.close(), so warning
filters managed via the cleanup stack (the warnings plugin's
catch_warnings context, in particular) are guaranteed active when GC

[paulzuradzki]

self-review

[paulzuradzki]

Explainer:

• Builds a PytestUnraisableExceptionWarning wrapper and emits that via warnings.warn. The user's filter is error::ResourceWarning.
• PytestUnraisableExceptionWarning is a pytest.PytestWarning subclass; no relationship to Python ResourceWarning in class hierarchy.
• So the user's filter doesn't match the wrapper. The wrapper gets emitted as a regular warning (just a stderr line in the warnings summary), not promoted to an error, and nothing fails the test.
• For the user's error::ResourceWarning filter to fail the suite on main, they'd have to also write error::PytestUnraisableExceptionWarning. This would be filtering on pytest's internal wrapping class. Leaky abstraction. They want to express "fail if a ResourceWarning leaks from a finalizer," not "fail if pytest's internal wrapper class fires."

[paulzuradzki]

Commit 4cf46 says, "if user said error::ResourceWarning, an unraisable ResourceWarning becomes a real raised ResourceWarning"

[paulzuradzki]

When collect_unraisable sees an unraisable whose exc_value is a Warning subclass instance, re-raise it directly instead of wrapping w/ pytest.PytestUnraisableExceptionWarning

[paulzuradzki]

Append meta.exc_value (the actual ResourceWarning instance) to errors (what eventually gets raise)

[paulzuradzki]

For Python 3.11+, attach meta.cause_msg as a PEP 678 note

[paulzuradzki]

Bug made possible by -- this PR -- moving GC and collect_unraisable from cleanup callback into pytest_unconfigure hook.

Guard checks: if our configure never ran, stash key is not there, and there's nothing to drain deque, so early return.

Else, this bug was possile:

• configure step crashes before def pytest_configure(config): ... config.stash[unraisable_exceptions] = deque
• stash never created
• unconfigure runs anyway
• -> tries to read missing stash
• -> KeyError

[paulzuradzki]

I noticed similar TODOs elsewhere.

AI explainer:

The exit code 1 comes from pytest's session-end unraisable handling, not from runpytest_subprocess() (that helper just reports the child's exit code). The leaked warning fires during session-end GC in pytest_unconfigure, after test_it has already been reported as passed, so it propagates to the top of the run instead of failing a specific test. Ideally we'd get failed=1 attributed to test_it; that needs allocation tracking and is out of scope here.

[paulzuradzki]

cc: @Zac-HD - this is the PR/GH Issue we chatted about at PyCon US sprints last Monday