how to print only failing test param in the test output

[skhal]

I have a value parameterized test with a long list of test params:

• 10+ test cases
• each test case consists of a few multi-line (10+ lines) @dataclasses

@dataclass(frozen=True):
class TestInput
  a: str
  # 10+ more fields

@dataclass(frozen=True):
class TestParam
  a: str
  # 5 more fields

@dataclass(frozen=True):
class TestWant
  a: str
  # 5 more fields

@pytest.mark.parametrize(
  "input,param,want",
  (
    pytest.param(
      TestInput(
        # 10+ args, one per line
      ),
      TestParam(
        # ~5 args, one per line
      ),
      TestWant(
        # ~5 args, one per line
      ),
      id = "first test case"
    ),
    # more test cases follow
  )
def test_foo(input, param, want):
  ...

Pytest prints the failed test case's params upon error along with the test_foo() function body, preceded by the full list of parameters @pytest.mark.parameterize(...) spanning many lines of stdout.

Is there a way to reduce the output of a failed test to

• only print the failed test case in @pytest.mark.parameterize(...)
• still print the body of test_foo() function

?

Demo

The demonstration includes comments in the test params - these should be output as well upon failure.

# file: ./param_test.py
import pytest


@pytest.mark.parametrize(
    "input,want",
    (
        # the parameters might be very long, spanning multiple lines
        pytest.param(1, 1, id="demo test passes"),
        # for example, the params might be dataclasses, e.g. TestInput and
        # TestWant.
        pytest.param(2, 3, id="demo test fails"),
        # more test data
        pytest.param(4, 4, id="demo test passes again"),
    ),
)
def test_demo(input, want):
    assert input == want

run:

% pytest --no-header ./param_test.py
=============================================================================== test session starts ================================================================================
collected 3 items                                                                                                                                                                   

param_test.py .F.                                                                                                                                                            [100%] 

===================================================================================== FAILURES =====================================================================================
____________________________________________________________________________ test_demo[demo test fails] ____________________________________________________________________________

input = 2, want = 3

    @pytest.mark.parametrize(
        "input,want",
        (
            # the parameters might be very long, spanning multiple lines
            pytest.param(1, 1, id="demo test passes"),
            # for example, the params might be dataclasses, e.g. TestInput and
            # TestWant.
            pytest.param(2, 3, id="demo test fails"),
            # more test data
            pytest.param(4, 4, id="demo test passes again"),
        ),
    )
    def test_demo(input, want):
>       assert input == want
E       assert 2 == 3

param_test.py:17: AssertionError
============================================================================= short test summary info ==============================================================================
FAILED param_test.py::test_demo[demo test fails] - assert 2 == 3
=========================================================================== 1 failed, 2 passed in 0.02s ============================================================================

want:

% pytest --no-header ./param_test.py
=============================================================================== test session starts ================================================================================
collected 3 items                                                                                                                                                                   

param_test.py .F.                                                                                                                                                            [100%] 

===================================================================================== FAILURES =====================================================================================
____________________________________________________________________________ test_demo[demo test fails] ____________________________________________________________________________

input = 2, want = 3

    @pytest.mark.parametrize(
        "input,want",
        (
            ...
            # for example, the params might be dataclasses, e.g. TestInput and
            # TestWant.
            pytest.param(2, 3, id="demo test fails"),
            ...
        ),
    )
    def test_demo(input, want):
>       assert input == want
E       assert 2 == 3

param_test.py:17: AssertionError
============================================================================= short test summary info ==============================================================================
FAILED param_test.py::test_demo[demo test fails] - assert 2 == 3
=========================================================================== 1 failed, 2 passed in 0.02s ============================================================================

[g4lb]

pytest prints the test source from the decorator. Move the cases to a variable, and the failure shows one decorator line, the body, and the failing values:

@pytest.mark.parametrize("input,want", CASES)

[skhal]

I tried that approach and it works except that now I have a variable CASES. The test file has a bunch of similar test_foo() functions, each focusing on a different aspect of the logic. I could have used FOO_CASES, BAR_CASES or even create a data-class to hold the test cases.

However, one of the things I find very useful in Pytest is co-location of the test body and test cases as in the following case:

@pytest.mark.parameterize(
  # all test cases are right here - easy to see, no jumps around
)
def test_foo():
  ...

Is there a way to customize the print of the decorator (i.e. examples)?

[g4lb]

Not with a built-in option: the default traceback prints the full function source, and that includes the decorator. --tb=short drops it, but also drops the body.