Add a unique test counter to the final report

[pomponchik]

What's the problem this feature will solve?

Hello! I'm a big fan of pytest. However, I ran into a problem when using pytest.mark.parametrize.

The fact is that each individual parameter for the test is always displayed as a whole test in the final report. For me, these reports are an important indicator, and it is important for me to distinguish unique tests from tests that are generated by the transmitted parameters.

Describe the solution you'd like

Ideally I would like to see both numbers: how many unique tests and how many of them in total, taking into account parameterized ones. This can be done both by default and when using a special command line option, for example --show_unique_tests_number.

[nicoddemus]

Hi @pomponchik ,

IMHO this feature is too specific to go into the core, but is something you can cook up yourself via a custom plugin (conftest.py).

One idea is to via pytest_report_teststatus count the "unique" items, and then pytest_terminal_summary to show a message with the unique items.

[RonnyPfannschmidt]

As far as pytest is concerned,each of the parameter combinations is a unique test item

So from the standpoint of the architecture, you want to count test "definitions"

Currently test definitions are only partially exposed in test Generation but not as Part of the collection tree

The final test items include parameterization

[pomponchik]

So from the standpoint of the architecture, you want to count test "definitions"

Yes, that's right, I want to count definitions.

[The-Compiler]

IMHO this might be an useful enough metric to show by default. It's nice to see that I have a couple thousand tests in my project, but I suspect this is for a big part from a couple of big parametrizations, so this would help to get a better feeling of the state of things (and also clarify things for beginners perhaps, who might be less aware of parametrizations).

[nicoddemus]

Not sure @The-Compiler, my gut feeling is that this will be confused to beginners specifically. But I don't care much either way TBH, so I'm changing my position here to -0 if somebody cares to implement this.

[pomponchik]

@nicoddemus I was a beginner some time ago, and it was the current behavior that was a little shock to me.

[Zac-HD]

Closing this because as per #12465 (comment) it can be implemented in an external plugin.