Check all warnings returned by pytest.warns() #9702

s-t-e-v-e-n-k · 2024-06-20T02:27:30Z

Change Summary

With pytest 8.x, pytest.warns() now emits all warnings, bump the required version of pytest and then assert that all of them are the expected warning, rather than filtering them out.

Related issue number

Fixes #9597

Checklist

The pull request title is a good summary of the changes - it will be used in the changelog
Unit tests for the changes exist
Tests pass on CI
Documentation reflects the changes where applicable
My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Selected Reviewer: @sydney-runkle

s-t-e-v-e-n-k · 2024-06-20T02:27:49Z

please review

codspeed-hq · 2024-06-20T02:32:33Z

CodSpeed Performance Report

Merging #9702 will not alter performance

_{Comparing s-t-e-v-e-n-k:check-all-warnings-with-pytest (43f6641) with main (5a77c5c)}

Summary

✅ 13 untouched benchmarks

sydney-runkle

Thanks for your awesome work here.

Looks like a few tests are still failing - perhaps we could still use context managers for each statement that warns, instead of catching all of the warnings in one place for each test. I think that might help iron out the current test failure as well!

sydney-runkle · 2024-06-20T14:45:04Z

tests/test_config.py

    def test_config_class_is_deprecated(self):
-        with warnings.catch_warnings():
-            # we need to explicitly ignore the other warning in pytest-8
-            # TODO: rewrite it to use two nested pytest.warns() when pytest-7 is no longer supported
-            warnings.simplefilter('ignore')
-            with pytest.warns(
-                PydanticDeprecatedSince20,
-                match='Support for class-based `config` is deprecated, use ConfigDict instead.',
-            ):
-
-                class Config(BaseConfig):
-                    pass
+        with pytest.warns(PydanticDeprecatedSince20) as all_warnings:
+
+            class Config(BaseConfig):
+                pass
+
+        assert len(all_warnings) == 2
+        expected_warnings = [
+            'BaseConfig is deprecated. Use the `pydantic.ConfigDict` instead',
+            'Support for class-based `config` is deprecated, use ConfigDict instead',
+        ]
+        assert [w.message.message for w in all_warnings] == expected_warnings

    def test_config_class_attributes_are_deprecated(self):


Looks like we're getting some failures for these two:

https://github.com/pydantic/pydantic/actions/runs/9590698692/job/26446460308?pr=9702

The changes look alright to me, though I haven't dug into the error cause.

Would this be a case where the nested context managers could help?

In some ways, I'd rather we go with that approach so that the error messages are localized and make more sense when contextualized directly above the error-throwing statement.

I doubt this one can be nested -- since typing-extensions=4.6.1 only emits one warning rather two, there'd still be a failure when the outer block doesn't throw a warning.

s-t-e-v-e-n-k · 2024-06-21T03:12:09Z

I am not sure what is causing the Windows CI failures. 😞

With pytest 8.x, pytest.warns() now emits all warnings, bump the required version of pytest and then assert that all of them are the expected warning, rather than filtering them out. Fixes pydantic#9597

sydney-runkle · 2024-06-21T13:32:30Z

@s-t-e-v-e-n-k,

Ah, looks like the lockfile and pyproject.toml were out of sync.

sydney-runkle

Thanks for your work on this.

I think there's still room to improve the readability of these tests if anyone is interested, though I think this is a good step forward in at least upgrading to pytest v8+

sydney-runkle · 2024-06-21T14:08:29Z

Here's a link to the pytest warnings docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

github-actions bot added the relnotes-fix Used for bugfixes. label Jun 20, 2024

pydantic-hooky bot added the ready for review label Jun 20, 2024

pydantic-hooky bot assigned sydney-runkle Jun 20, 2024

s-t-e-v-e-n-k force-pushed the check-all-warnings-with-pytest branch from c8af877 to 3dad616 Compare June 20, 2024 02:29

sydney-runkle suggested changes Jun 20, 2024

View reviewed changes

pydantic-hooky bot added awaiting author revision awaiting changes from the PR author and removed ready for review labels Jun 20, 2024

pydantic-hooky bot assigned s-t-e-v-e-n-k and unassigned sydney-runkle Jun 20, 2024

s-t-e-v-e-n-k force-pushed the check-all-warnings-with-pytest branch from 3dad616 to 2c7052f Compare June 21, 2024 03:08

Check all warnings returned by pytest.warns()

195f8fc

With pytest 8.x, pytest.warns() now emits all warnings, bump the required version of pytest and then assert that all of them are the expected warning, rather than filtering them out. Fixes pydantic#9597

s-t-e-v-e-n-k force-pushed the check-all-warnings-with-pytest branch from 2c7052f to 195f8fc Compare June 21, 2024 03:17

s-t-e-v-e-n-k requested a review from sydney-runkle June 21, 2024 03:17

sydney-runkle added 8 commits June 21, 2024 08:51

maybe fixing content hash

5095f87

try another update

d2929f1

try setting encoding

1f75d19

export maybe

6bebe3e

export to env

77f5c09

relocking everything

565ef55

other updates

76fd771

try more conservative deps update

43f6641

sydney-runkle added relnotes-packaging Used for dependency changes. and removed relnotes-fix Used for bugfixes. labels Jun 21, 2024

sydney-runkle approved these changes Jun 21, 2024

View reviewed changes

sydney-runkle merged commit 09bb639 into pydantic:main Jun 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Check all warnings returned by pytest.warns() #9702

Check all warnings returned by pytest.warns() #9702

Uh oh!

s-t-e-v-e-n-k commented Jun 20, 2024 •

edited by pydantic-hooky bot

Loading

Uh oh!

s-t-e-v-e-n-k commented Jun 20, 2024

Uh oh!

codspeed-hq bot commented Jun 20, 2024 •

edited

Loading

Uh oh!

sydney-runkle left a comment

Uh oh!

sydney-runkle Jun 20, 2024

Uh oh!

sydney-runkle Jun 20, 2024

Uh oh!

sydney-runkle Jun 20, 2024

Uh oh!

s-t-e-v-e-n-k Jun 21, 2024

Uh oh!

s-t-e-v-e-n-k commented Jun 21, 2024

Uh oh!

sydney-runkle commented Jun 21, 2024

Uh oh!

sydney-runkle left a comment

Uh oh!

sydney-runkle commented Jun 21, 2024

Uh oh!

Uh oh!

Uh oh!

Check all warnings returned by pytest.warns() #9702

Check all warnings returned by pytest.warns() #9702

Uh oh!

Conversation

s-t-e-v-e-n-k commented Jun 20, 2024 • edited by pydantic-hooky bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Summary

Related issue number

Checklist

Uh oh!

s-t-e-v-e-n-k commented Jun 20, 2024

Uh oh!

codspeed-hq bot commented Jun 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #9702 will not alter performance

Summary

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

sydney-runkle Jun 20, 2024

Choose a reason for hiding this comment

Uh oh!

sydney-runkle Jun 20, 2024

Choose a reason for hiding this comment

Uh oh!

sydney-runkle Jun 20, 2024

Choose a reason for hiding this comment

Uh oh!

s-t-e-v-e-n-k Jun 21, 2024

Choose a reason for hiding this comment

Uh oh!

s-t-e-v-e-n-k commented Jun 21, 2024

Uh oh!

sydney-runkle commented Jun 21, 2024

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

sydney-runkle commented Jun 21, 2024

Uh oh!

Uh oh!

s-t-e-v-e-n-k commented Jun 20, 2024 •

edited by pydantic-hooky bot

Loading

codspeed-hq bot commented Jun 20, 2024 •

edited

Loading