deprecate non-cm raises,warns&deprecated call + add paramspec

[jakkdl]

deprecation

Deprecates the legacy callable forms for pytest.raises, pytest.warns and pytest.deprecated_call, and replaces all instances in the code base of them aside from code specifically for testing them.

paramspec

pytest.raises, warns & deprecated_call previously typed *args and **kwargs as Any in the legacy callable form, so this did not raise errors:

def foo(x: int) -> None:
    raise ValueError
raises(ValueError, foo, None)

but now it will give call-overload.

It also makes it possible to pass func as a kwarg, which the type hints previously showed as possible, but it didn't work.

It's possible that func (and the expected type?) should be pos-only, as this looks quite weird:

raises(1, 2, kwarg1=3, func=my_func, kwarg2=4, expected_exception=ValueError)

but if somebody is dynamically generating parameters to send to raises then we probably shouldn't ban it needlessly; and we can't make func pos-only without making expected_exception pos-only, and that could break backwards compatibility.

• Include documentation when adding new features.
• Include new tests or update existing tests when applicable.
• Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself.
• Create a new changelog file in the changelog folder, with a name like <ISSUE NUMBER>.<TYPE>.rst. See changelog/README.rst for details.

Noticed while working on #13192

[RonnyPfannschmidt]

I believe the key issue is that back when the api was initially created, neither typing nor POS only arguments where a thing

In hindsight, both func and expected exceptions should be POS only arguments

Some kind of deprecating overload would be needed (no cluesif typing supports anyof that)

I'm thinking whether we should just downright deprecate the call form

Back in python 2.5 times when we had no with statement there only was a call form and a exec form (multiline stringto eval)

Now that soon python 3.8 support is dropped we perhaps ought to just go context only and plan accordingly

[jakkdl]

I believe the key issue is that back when the api was initially created, neither typing nor POS only arguments where a thing

In hindsight, both func and expected exceptions should be POS only arguments

I could make them pos-only for RaisesExc and RaisesGroup in #13192 to start with, so we only need to do deprecation on pytest.raises itself.

Some kind of deprecating overload would be needed (no cluesif typing supports anyof that)

@warnings.deprecated does indeed allow for deprecating individual overloads.

I'm thinking whether we should just downright deprecate the call form

Back in python 2.5 times when we had no with statement there only was a call form and a exec form (multiline stringto eval)

Now that soon python 3.8 support is dropped we perhaps ought to just go context only and plan accordingly

the docs does say

This form was the original pytest.raises() API, developed before the with statement was added to the Python language. Nowadays, this form is rarely used, with the context-manager form (using with) being considered more readable. Nonetheless, this form is fully supported and not deprecated in any way.

(emphasis mine) which kind of sounds like us wanting to support it indefinitely, but I have never seen the form used outside of the pytest repo itself, so I think we should be plenty fine deprecating it.

[jakkdl]

"well, let's just deprecate them then!"
...

thankfully I had LLM generate a regex that did a lot of the replacements, but, uh, fun times.

I don't think there's much to gain from deprecating pytest.raises(expected_exception=ValueError)?

TODO:

• Changelog
• update PR title & description (lol)
• update docs

[jakkdl]

the other stuff causing <100% diff coverage is me editing currently-uncovered code

[nicoddemus]

Thanks @jakkdl for tackling this.

Left comments mostly related to the docs.

However I wonder if we really should deprecate this functionality?

• We never said it would deprecated.
• Changing to the context manager is non-trivial to do, which will cause a lot of friction in older codebases.
• The code to support the legacy form seems pretty minimal to me, so we gain little by removing the support.

So my vote on this is -0, as I think the disruption/friction outweighs the small benefit of removing the supporting code, but if everybody else thinks this is worthwhile, then I'm OK with it too.

[nicoddemus]

warnings.deprecated will emit a warning at runtime in addition to be used by type checkers, but not for Python <= 3.12, which IIUC is a problem then.

Seems like the only solution is to always generate the runtime warning ourselves, and only rely on the deprecated decorator when TYPE_CHECKING?

[jakkdl]

hm, since this PR does manually raise the warning everywhere we should be getting double warnings on 3.13 ... and either the tests are not catching it or the second one is getting suppressed.

[jakkdl]

Ah, @deprecated is only being used together with @overload in this PR; so it is solely used for TYPE_CHECKING. I'll clarify that with comments/docstring in case somebody wants to use it more in the future

[jakkdl]

• The code to support the legacy form seems pretty minimal to me, so we gain little by removing the support.

It's not a lot of supporting code at its face, but it took me a while to figure out gc troubles in 13192#2c8cd and raises could entirely be removed in place of a from ... export RaisesExc as raises after #13192.

It's also a question of whether to keep it maintained with RaisesExc - e.g. should it now accept the check parameter? If so, that will break code:

def my_fun(check):
    ...

def test_my_fun():
    pytest.raises(my_fun, check=5)

if we wanted to fully embrace it then maybe RaisesGroup should have a callable form, but that'd be kinda silly.

• Changing to the context manager is non-trivial to do, which will cause a lot of friction in older codebases.

it'd be nice to get a sense of how prevalent it is, I've personally never encountered it outside of the pytest code base itself.

So my vote on this is -0, as I think the disruption/friction outweighs the small benefit of removing the supporting code, but if everybody else thinks this is worthwhile, then I'm OK with it too.

I agree that it's not a slam-dunk, and if we deprecate it we should make sure to have a long deprecation cycle. Heck, we could try deprecating it and if there's complaints we consider reversing. (futuredeprecationwarning? and/or a message explicitly saying so)

[RonnyPfannschmidt]

perhaps before deprecating we should see to have tools like ruff supply a code fix and/or have a ast-grep receipt to do the code fix

the code change itself is not that hard, but its a massive friction without a tool that does it

[jakkdl]

perhaps before deprecating we should see to have tools like ruff supply a code fix and/or have a ast-grep receipt to do the code fix

the code change itself is not that hard, but its a massive friction without a tool that does it

I like that, I'll go open some issues.
In order not to have this PR stall forever, let's do PendingDeprecationWarning for now.

edit: FutureDeprecationWarning -> PendingDeprecationWarning

[RonnyPfannschmidt]

Let's see if the future warning triggers any issues for users

I'm on board however python warning is quite some unexpected fun sometimes

[jakkdl]

hm, while the default python filter will hide PendingDeprecationWarning, it appears pytest does not. So having it be a PendingDeprecationWarning won't really change the direct impact to users, although we're signaling that they don't need to rush with a fix.

If we do want to hold off on even that then I should split this PR and we can merge the test changes + paramspec on their own

[RonnyPfannschmidt]

Let's put it in for now

We can put it on defer if there's a problematic impact most recent user's shouldn't be affected

[jakkdl]

codecov diff complaints are solely from me changing callable raises to cm-raises in code that's not covered on main/ - technically adding more uncovered lines