Distinct failure tracebacks of ExceptionGroup are not printed by pytest

[subatiq]

I'm getting a notion about 4 distinct errors from the test and nothing else.

---

Tested on python 3.7.5 and 3.10.5. Also on pytest 6.2.4 and 7.1.2

Running the pytest from the terminal. In this case like that

python3 -m pytest -vv tests/unit/unifiers/test_frame_scaling.py

The output:

platform linux -- Python 3.7.5, pytest-7.1.2, pluggy-1.0.0                                                                                                                                                                                                                                                                    
rootdir: /app/core/tests, configfile: pytest.ini                                                                                                                                                                                                                                                                              
plugins: hypothesis-6.54.0, cov-3.0.0                                                                                                                                                                                                                                                                                         
collected 1 item                                                                                                                                                                                                                                                                                                              
                                                                                                                                                                                                                                                                                                                              
core/tests/unit/unifiers/test_frame_scaling.py F                                                                                                                                                                                                                                                                       [100%] 
                                                                                                                                                                                                                                                                                                                              
================== FAILURES ==================
test_frame_persists_aspect_ratio_after_scale 
                                                                                                                                                                                                                                                                                                                              
    @given(...)                                                                                                                                                                                                                                                                                                               
>   def test_frame_persists_aspect_ratio_after_scale(resolution_in: ImageResolution, resolution_out: ImageResolution):                                                                                                                                                                                                        
E   exceptiongroup.ExceptionGroup: Hypothesis found 4 distinct failures. (4 sub-exceptions)                                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                                                                                              
core/tests/unit/unifiers/test_frame_scaling.py:7: ExceptionGroup                                                                                                                                                                                                                                                              
generated xml file: /app/reports/report.xml 
=========== short test summary info ============
FAILED core/tests/unit/unifiers/test_frame_scaling.py::test_frame_persists_aspect_ratio_after_scale - exceptiongroup.ExceptionGroup: Hypothesis found 4 distinct failures. (4 sub-exceptions)                                                                                                                                 
=========== 1 failed in 1.28s ===========

Desired outcome: unwrapped tracebacks for every sub-exception

[honno]

This is due to #3308, where tl;dr a new standard feature has been adopted (ExceptionGroup), but pytest has to catch up a bit still.

For now, you can use pytest's --tb=native flag to see the tracebacks for each sub-exception. In the future pytest should show these by default pytest-dev/pytest#9159

[subatiq]

Worked like a charm. Thank you!

[honno]

Thanks for opening, this seems like it'll trip up a few folks so I'll pin it.

[Zac-HD]

I also thought that I'd disabled multi-bug reporting for these cases where it doesn't work 🤔

Needs more tests + a fix on our end too.

[Zac-HD]

Fixed in our latest version - thanks for the report @subatiq!