Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some tests are failing #21

Closed
fabaff opened this issue Mar 23, 2020 · 6 comments
Closed

Some tests are failing #21

fabaff opened this issue Mar 23, 2020 · 6 comments

Comments

@fabaff
Copy link

fabaff commented Mar 23, 2020 

[fab@1234 repos]$ git clone git@github.com:fabaff/pytest-subtests.git
Cloning into 'pytest-subtests'...
remote: Enumerating objects: 25, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 166 (delta 5), reused 12 (delta 3), pack-reused 141
Receiving objects: 100% (166/166), 39.34 KiB | 592.00 KiB/s, done.
Resolving deltas: 100% (74/74), done.
[fab@1234 repos]$ cd pytest-subtests/
[fab@1234 pytest-subtests]$ python3 -m venv .
[fab@1234 pytest-subtests]$ source bin/activate
(pytest-subtests) [fab@1234 pytest-subtests]$ python3 setup.py develop
running develop
running egg_info
creating pytest_subtests.egg-info
writing pytest_subtests.egg-info/PKG-INFO
writing dependency_links to pytest_subtests.egg-info/dependency_links.txt
writing entry points to pytest_subtests.egg-info/entry_points.txt
writing requirements to pytest_subtests.egg-info/requires.txt
writing top-level names to pytest_subtests.egg-info/top_level.txt
writing manifest file 'pytest_subtests.egg-info/SOURCES.txt'
writing manifest file 'pytest_subtests.egg-info/SOURCES.txt'
running build_ext
Creating /home/fab/Documents/repos/pytest-subtests/lib/python3.7/site-packages/pytest-subtests.egg-link (link to .)
Adding pytest-subtests 0.3.1.dev1+gc5442e3 to easy-install.pth file

Installed /home/fab/Documents/repos/pytest-subtests
Processing dependencies for pytest-subtests==0.3.1.dev1+gc5442e3
[...]
Finished processing dependencies for pytest-subtests==0.3.1.dev1+gc5442e3

(pytest-subtests) [fab@1234 pytest-subtests]$ pytest-3 -v tests
/usr/lib/python3.7/site-packages/trio/_core/_multierror.py:450: RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing trio's custom handler, but this means MultiErrors will not show full tracebacks.
  category=RuntimeWarning
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /home/fab/Documents/repos/pytest-subtests
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0
collecting ... 

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestFixture.test_simple_terminal_normal[normal] ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestFixture object at 0x7f12c26c0fd0>, simple_script = None, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal0')>
mode = 'normal'

    def test_simple_terminal_normal(self, simple_script, testdir, mode):
        if mode == "normal":
            result = testdir.runpytest()
            expected_lines = ["collected 1 item"]
        else:
            pytest.importorskip("xdist")
            result = testdir.runpytest("-n1")
            expected_lines = ["gw0 [1]"]
    
        expected_lines += [
            "* test_foo [[]custom[]] (i=1) *",
            "* test_foo [[]custom[]] (i=3) *",
            "* 2 failed, 1 passed in *",
        ]
>       result.stdout.fnmatch_lines(expected_lines)
E       Failed: nomatch: 'collected 1 item'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal0'
E           and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: ''
E           and: ''
E           and: '―――――――――――――――――――――――――― ERROR at setup of test_foo ――――――――――――――――――――――――――'
E           and: 'file /tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal0/test_simple_terminal_normal.py, line 1'
E           and: '  def test_foo(subtests):'
E           and: "E       fixture 'subtests' not found"
E           and: '>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave'
E           and: ">       use 'pytest --fixtures [testpath]' for help on them."
E           and: ''
E           and: '/tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal0/test_simple_terminal_normal.py:1'
E           and: '\r                                                                 \x1b[32m100% \x1b[0m\x1b[40m██████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.08s):'
E           and: '\x1b[31m       1 error\x1b[0m'
E           and: ''
E       remains unmatched: 'collected 1 item'

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:37: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal0
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0


―――――――――――――――――――――――――― ERROR at setup of test_foo ――――――――――――――――――――――――――
file /tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal0/test_simple_terminal_normal.py, line 1
  def test_foo(subtests):
E       fixture 'subtests' not found
>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal0/test_simple_terminal_normal.py:1
                                                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.08s):
       1 error

 tests/test_subtests.py::TestFixture.test_simple_terminal_normal[normal] ⨯                                                                                           5% ▌         
 tests/test_subtests.py::TestFixture.test_simple_terminal_normal[xdist] s                                                                                           11% █▏        

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestFixture.test_simple_terminal_verbose[normal] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestFixture object at 0x7f12c1a71090>, simple_script = None, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose0')>
mode = 'normal'

    def test_simple_terminal_verbose(self, simple_script, testdir, mode):
        if mode == "normal":
            result = testdir.runpytest("-v")
            expected_lines = [
                "*collected 1 item",
                "test_simple_terminal_verbose.py::test_foo PASSED *100%*",
                "test_simple_terminal_verbose.py::test_foo FAILED *100%*",
                "test_simple_terminal_verbose.py::test_foo PASSED *100%*",
                "test_simple_terminal_verbose.py::test_foo FAILED *100%*",
                "test_simple_terminal_verbose.py::test_foo PASSED *100%*",
                "test_simple_terminal_verbose.py::test_foo PASSED *100%*",
            ]
        else:
            pytest.importorskip("xdist")
            result = testdir.runpytest("-n1", "-v")
            expected_lines = [
                "gw0 [1]",
                "*gw0*100%* test_simple_terminal_verbose.py::test_foo*",
                "*gw0*100%* test_simple_terminal_verbose.py::test_foo*",
                "*gw0*100%* test_simple_terminal_verbose.py::test_foo*",
                "*gw0*100%* test_simple_terminal_verbose.py::test_foo*",
                "*gw0*100%* test_simple_terminal_verbose.py::test_foo*",
                "*gw0*100%* test_simple_terminal_verbose.py::test_foo*",
            ]
    
        expected_lines += [
            "* test_foo [[]custom[]] (i=1) *",
            "* test_foo [[]custom[]] (i=3) *",
            "* 2 failed, 1 passed in *",
        ]
>       result.stdout.fnmatch_lines(expected_lines)
E       Failed: nomatch: '*collected 1 item'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: 'cachedir: .pytest_cache'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose0'
E           and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: 'collecting ... '
E           and: ''
E           and: '―――――――――――――――――――――――――― ERROR at setup of test_foo ――――――――――――――――――――――――――'
E           and: 'file /tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose0/test_simple_terminal_verbose.py, line 1'
E           and: '  def test_foo(subtests):'
E           and: "E       fixture 'subtests' not found"
E           and: '>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave'
E           and: ">       use 'pytest --fixtures [testpath]' for help on them."
E           and: ''
E           and: '/tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose0/test_simple_terminal_verbose.py:1'
E           and: '\r                                                                 \x1b[32m100% \x1b[0m\x1b[40m██████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.06s):'
E           and: '\x1b[31m       1 error\x1b[0m'
E           and: ''
E       remains unmatched: '*collected 1 item'

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:69: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose0
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0
collecting ... 

―――――――――――――――――――――――――― ERROR at setup of test_foo ――――――――――――――――――――――――――
file /tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose0/test_simple_terminal_verbose.py, line 1
  def test_foo(subtests):
E       fixture 'subtests' not found
>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose0/test_simple_terminal_verbose.py:1
                                                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.06s):
       1 error

 tests/test_subtests.py::TestFixture.test_simple_terminal_verbose[normal] ⨯                                                                                         16% █▋        
 tests/test_subtests.py::TestFixture.test_simple_terminal_verbose[xdist] s                                                                                          21% ██▏       

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestFixture.test_skip[normal] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestFixture object at 0x7f12cdfb7990>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_skip0')>, mode = 'normal'

    def test_skip(self, testdir, mode):
        testdir.makepyfile(
            """
            import pytest
            def test_foo(subtests):
                for i in range(5):
                    with subtests.test(msg="custom", i=i):
                        if i % 2 == 0:
                            pytest.skip('even number')
        """
        )
        if mode == "normal":
            result = testdir.runpytest()
            expected_lines = ["collected 1 item"]
        else:
            pytest.importorskip("xdist")
            result = testdir.runpytest("-n1")
            expected_lines = ["gw0 [1]"]
        expected_lines += ["* 1 passed, 3 skipped in *"]
>       result.stdout.fnmatch_lines(expected_lines)
E       Failed: nomatch: 'collected 1 item'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_skip0'
E           and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: ''
E           and: ''
E           and: '―――――――――――――――――――――――――― ERROR at setup of test_foo ――――――――――――――――――――――――――'
E           and: 'file /tmp/pytest-of-fab/pytest-4/test_skip0/test_skip.py, line 2'
E           and: '  def test_foo(subtests):'
E           and: "E       fixture 'subtests' not found"
E           and: '>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave'
E           and: ">       use 'pytest --fixtures [testpath]' for help on them."
E           and: ''
E           and: '/tmp/pytest-of-fab/pytest-4/test_skip0/test_skip.py:2'
E           and: '\r                                                                 \x1b[32m100% \x1b[0m\x1b[40m██████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.06s):'
E           and: '\x1b[31m       1 error\x1b[0m'
E           and: ''
E       remains unmatched: 'collected 1 item'

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:90: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_skip0
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0


―――――――――――――――――――――――――― ERROR at setup of test_foo ――――――――――――――――――――――――――
file /tmp/pytest-of-fab/pytest-4/test_skip0/test_skip.py, line 2
  def test_foo(subtests):
E       fixture 'subtests' not found
>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-fab/pytest-4/test_skip0/test_skip.py:2
                                                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.06s):
       1 error

 tests/test_subtests.py::TestFixture.test_skip[normal] ⨯                                                                                                            26% ██▋       
 tests/test_subtests.py::TestFixture.test_skip[xdist] s                                                                                                             32% ███▎      
 tests/test_subtests.py::TestSubTest.test_simple_terminal_normal[unittest] ✓                                                                                        37% ███▊      

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestSubTest.test_simple_terminal_normal[pytest-normal] ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestSubTest object at 0x7f12cdf6a750>, simple_script = local('/tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal3/test_simple_terminal_normal.py')
testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal3')>, runner = 'pytest-normal'

    @pytest.mark.parametrize("runner", ["unittest", "pytest-normal", "pytest-xdist"])
    def test_simple_terminal_normal(self, simple_script, testdir, runner):
    
        if runner == "unittest":
            result = testdir.run(sys.executable, simple_script)
            result.stderr.fnmatch_lines(
                [
                    "FAIL: test_foo (__main__.T) [custom] (i=1)",
                    "AssertionError: 1 != 0",
                    "FAIL: test_foo (__main__.T) [custom] (i=3)",
                    "AssertionError: 1 != 0",
                    "Ran 1 test in *",
                    "FAILED (failures=2)",
                ]
            )
        else:
            if runner == "pytest-normal":
                result = testdir.runpytest(simple_script)
                expected_lines = ["collected 1 item"]
            else:
                pytest.importorskip("xdist")
                result = testdir.runpytest(simple_script, "-n1")
                expected_lines = ["gw0 [1]"]
            result.stdout.fnmatch_lines(
                expected_lines
                + [
                    "* T.test_foo [[]custom[]] (i=1) *",
                    "E  * AssertionError: 1 != 0",
                    "* T.test_foo [[]custom[]] (i=3) *",
                    "E  * AssertionError: 1 != 0",
>                   "* 2 failed, 1 passed in *",
                ]
            )
E           Failed: nomatch: 'collected 1 item'
E               and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E               and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E               and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal3'
E               and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E               and: ''
E               and: ''
E               and: '―――――――――――――――――――――――――――――――――― T.test_foo ――――――――――――――――――――――――――――――――――'
E               and: ''
E               and: 'self = <test_simple_terminal_normal.T testMethod=test_foo>'
E               and: ''
E               and: '    def test_foo(self):'
E               and: '        for i in range(5):'
E               and: '            with self.subTest(msg="custom", i=i):'
E               and: '>               self.assertEqual(i % 2, 0)'
E               and: 'E               AssertionError: 1 != 0'
E               and: ''
E               and: 'test_simple_terminal_normal.py:8: AssertionError'
E               and: '\r'
E               and: '\r \x1b[36m\x1b[0mtest_simple_terminal_normal.py\x1b[0m \x1b[31m⨯\x1b[0m                                \x1b[31m100% \x1b[0m\x1b[40m\x1b[31m█\x1b[0m\x1b[40m\x1b[31m█████████\x1b[0m'
E               and: '===Flaky Test Report==='
E               and: ''
E               and: ''
E               and: '===End Flaky Test Report==='
E               and: ''
E               and: 'Results (0.08s):'
E               and: '\x1b[31m       1 failed\x1b[0m'
E               and: '         - \x1b[36m\x1b[0mtest_simple_terminal_normal.py\x1b[0m:5 \x1b[31mT.test_foo\x1b[0m'
E               and: ''
E           remains unmatched: 'collected 1 item'

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:146: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_simple_terminal_normal3
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0


―――――――――――――――――――――――――――――――――― T.test_foo ――――――――――――――――――――――――――――――――――

self = <test_simple_terminal_normal.T testMethod=test_foo>

    def test_foo(self):
        for i in range(5):
            with self.subTest(msg="custom", i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_simple_terminal_normal.py:8: AssertionError

 test_simple_terminal_normal.py ⨯                                100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.08s):
       1 failed
         - test_simple_terminal_normal.py:5 T.test_foo

 tests/test_subtests.py::TestSubTest.test_simple_terminal_normal[pytest-normal] ⨯                                                                                   42% ████▎     
 tests/test_subtests.py::TestSubTest.test_simple_terminal_normal[pytest-xdist] s                                                                                    47% ████▊     
 tests/test_subtests.py::TestSubTest.test_simple_terminal_verbose[unittest] ✓                                                                                       53% █████▍    

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestSubTest.test_simple_terminal_verbose[pytest-normal] ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestSubTest object at 0x7f12cdefdad0>, simple_script = local('/tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose3/test_simple_terminal_verbose.py')
testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose3')>, runner = 'pytest-normal'

    @pytest.mark.parametrize("runner", ["unittest", "pytest-normal", "pytest-xdist"])
    def test_simple_terminal_verbose(self, simple_script, testdir, runner):
    
        if runner == "unittest":
            result = testdir.run(sys.executable, simple_script, "-v")
            result.stderr.fnmatch_lines(
                [
                    "test_foo (__main__.T) ... ",
                    "FAIL: test_foo (__main__.T) [custom] (i=1)",
                    "AssertionError: 1 != 0",
                    "FAIL: test_foo (__main__.T) [custom] (i=3)",
                    "AssertionError: 1 != 0",
                    "Ran 1 test in *",
                    "FAILED (failures=2)",
                ]
            )
        else:
            if runner == "pytest-normal":
                result = testdir.runpytest(simple_script, "-v")
                expected_lines = [
                    "*collected 1 item",
                    "test_simple_terminal_verbose.py::T::test_foo FAILED *100%*",
                    "test_simple_terminal_verbose.py::T::test_foo FAILED *100%*",
                    "test_simple_terminal_verbose.py::T::test_foo PASSED *100%*",
                ]
            else:
                pytest.importorskip("xdist")
                result = testdir.runpytest(simple_script, "-n1", "-v")
                expected_lines = [
                    "gw0 [1]",
                    "*gw0*100%* FAILED test_simple_terminal_verbose.py::T::test_foo*",
                    "*gw0*100%* FAILED test_simple_terminal_verbose.py::T::test_foo*",
                    "*gw0*100%* PASSED test_simple_terminal_verbose.py::T::test_foo*",
                ]
            result.stdout.fnmatch_lines(
                expected_lines
                + [
                    "* T.test_foo [[]custom[]] (i=1) *",
                    "E  * AssertionError: 1 != 0",
                    "* T.test_foo [[]custom[]] (i=3) *",
                    "E  * AssertionError: 1 != 0",
>                   "* 2 failed, 1 passed in *",
                ]
            )
E           Failed: nomatch: '*collected 1 item'
E               and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E               and: 'cachedir: .pytest_cache'
E               and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E               and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose3'
E               and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E               and: 'collecting ... '
E               and: ''
E               and: '―――――――――――――――――――――――――――――――――― T.test_foo ――――――――――――――――――――――――――――――――――'
E               and: ''
E               and: 'self = <test_simple_terminal_verbose.T testMethod=test_foo>'
E               and: ''
E               and: '    def test_foo(self):'
E               and: '        for i in range(5):'
E               and: '            with self.subTest(msg="custom", i=i):'
E               and: '>               self.assertEqual(i % 2, 0)'
E               and: 'E               AssertionError: 1 != 0'
E               and: ''
E               and: 'test_simple_terminal_verbose.py:8: AssertionError'
E               and: '\r'
E               and: '\r \x1b[36mtest_simple_terminal_verbose.py\x1b[0m::T.test_foo\x1b[0m \x1b[31m⨯\x1b[0m                   \x1b[31m100% \x1b[0m\x1b[40m\x1b[31m█\x1b[0m\x1b[40m\x1b[31m█████████\x1b[0m'
E               and: '===Flaky Test Report==='
E               and: ''
E               and: ''
E               and: '===End Flaky Test Report==='
E               and: ''
E               and: 'Results (0.08s):'
E               and: '\x1b[31m       1 failed\x1b[0m'
E               and: '         - \x1b[36m\x1b[0mtest_simple_terminal_verbose.py\x1b[0m:5 \x1b[31mT.test_foo\x1b[0m'
E               and: ''
E           remains unmatched: '*collected 1 item'

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:191: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_simple_terminal_verbose3
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0
collecting ... 

―――――――――――――――――――――――――――――――――― T.test_foo ――――――――――――――――――――――――――――――――――

self = <test_simple_terminal_verbose.T testMethod=test_foo>

    def test_foo(self):
        for i in range(5):
            with self.subTest(msg="custom", i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_simple_terminal_verbose.py:8: AssertionError

 test_simple_terminal_verbose.py::T.test_foo ⨯                   100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.08s):
       1 failed
         - test_simple_terminal_verbose.py:5 T.test_foo

 tests/test_subtests.py::TestSubTest.test_simple_terminal_verbose[pytest-normal] ⨯                                                                                  58% █████▊    
 tests/test_subtests.py::TestSubTest.test_simple_terminal_verbose[pytest-xdist] s                                                                                   63% ██████▍   
 tests/test_subtests.py::TestSubTest.test_skip[unittest] ✓                                                                                                          68% ██████▉   
 tests/test_subtests.py::TestSubTest.test_skip[pytest-normal] x                                                                                                     74% ███████▍  
 tests/test_subtests.py::TestSubTest.test_skip[pytest-xdist] x                                                                                                      79% ███████▉  

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestCapture.test_capturing ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestCapture object at 0x7f12c14c03d0>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_capturing0')>

    def test_capturing(self, testdir):
        self.create_file(testdir)
        result = testdir.runpytest()
        result.stdout.fnmatch_lines(
            [
                "*__ test (i='A') __*",
                "*Captured stdout call*",
                "hello stdout A",
                "*Captured stderr call*",
                "hello stderr A",
                "*__ test (i='B') __*",
                "*Captured stdout call*",
                "hello stdout B",
                "*Captured stderr call*",
                "hello stderr B",
                "*__ test __*",
                "*Captured stdout call*",
                "start test",
>               "end test",
            ]
        )
E       Failed: nomatch: "*__ test (i='A') __*"
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_capturing0'
E           and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: ''
E           and: ''
E           and: '―――――――――――――――――――――――――――― ERROR at setup of test ――――――――――――――――――――――――――――'
E           and: 'file /tmp/pytest-of-fab/pytest-4/test_capturing0/test_capturing.py, line 2'
E           and: '  def test(subtests):'
E           and: "E       fixture 'subtests' not found"
E           and: '>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave'
E           and: ">       use 'pytest --fixtures [testpath]' for help on them."
E           and: ''
E           and: '/tmp/pytest-of-fab/pytest-4/test_capturing0/test_capturing.py:2'
E           and: '\r                                                                 \x1b[32m100% \x1b[0m\x1b[40m██████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.06s):'
E           and: '\x1b[31m       1 error\x1b[0m'
E           and: ''
E       remains unmatched: "*__ test (i='A') __*"

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:266: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_capturing0
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0


―――――――――――――――――――――――――――― ERROR at setup of test ――――――――――――――――――――――――――――
file /tmp/pytest-of-fab/pytest-4/test_capturing0/test_capturing.py, line 2
  def test(subtests):
E       fixture 'subtests' not found
>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-fab/pytest-4/test_capturing0/test_capturing.py:2
                                                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.06s):
       1 error

 tests/test_subtests.py::TestCapture.test_capturing ⨯                                                                                                               84% ████████▌ 

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestCapture.test_no_capture ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestCapture object at 0x7f12c14d5810>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_no_capture0')>

    def test_no_capture(self, testdir):
        self.create_file(testdir)
        result = testdir.runpytest("-s")
        result.stdout.fnmatch_lines(
            [
                "start test",
                "hello stdout A",
                "Fhello stdout B",
                "Fend test",
                "*__ test (i='A') __*",
                "*__ test (i='B') __*",
>               "*__ test __*",
            ]
        )
E       Failed: nomatch: 'start test'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_no_capture0'
E           and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: ''
E           and: ''
E           and: '―――――――――――――――――――――――――――― ERROR at setup of test ――――――――――――――――――――――――――――'
E           and: 'file /tmp/pytest-of-fab/pytest-4/test_no_capture0/test_no_capture.py, line 2'
E           and: '  def test(subtests):'
E           and: "E       fixture 'subtests' not found"
E           and: '>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave'
E           and: ">       use 'pytest --fixtures [testpath]' for help on them."
E           and: ''
E           and: '/tmp/pytest-of-fab/pytest-4/test_no_capture0/test_no_capture.py:2'
E           and: '\r                                                                 \x1b[32m100% \x1b[0m\x1b[40m██████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.06s):'
E           and: '\x1b[31m       1 error\x1b[0m'
E           and: ''
E       remains unmatched: 'start test'

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:281: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_no_capture0
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0


―――――――――――――――――――――――――――― ERROR at setup of test ――――――――――――――――――――――――――――
file /tmp/pytest-of-fab/pytest-4/test_no_capture0/test_no_capture.py, line 2
  def test(subtests):
E       fixture 'subtests' not found
>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-fab/pytest-4/test_no_capture0/test_no_capture.py:2
                                                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.06s):
       1 error

 tests/test_subtests.py::TestCapture.test_no_capture ⨯                                                                                                              89% ████████▉ 

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestCapture.test_capture_with_fixture[capsys] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestCapture object at 0x7f12c1372ad0>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_capture_with_fixture0')>, fixture = 'capsys'

    @pytest.mark.parametrize("fixture", ["capsys", "capfd"])
    def test_capture_with_fixture(self, testdir, fixture):
        testdir.makepyfile(
            r"""
            import sys
    
            def test(subtests, {fixture}):
                print('start test')
    
                with subtests.test(i='A'):
                    print("hello stdout A")
                    print("hello stderr A", file=sys.stderr)
    
                out, err = {fixture}.readouterr()
                assert out == 'start test\nhello stdout A\n'
                assert err == 'hello stderr A\n'
        """.format(
                fixture=fixture
            )
        )
        result = testdir.runpytest()
        result.stdout.fnmatch_lines(
>           ["*1 passed*",]
        )
E       Failed: nomatch: '*1 passed*'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_capture_with_fixture0'
E           and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: ''
E           and: ''
E           and: '―――――――――――――――――――――――――――― ERROR at setup of test ――――――――――――――――――――――――――――'
E           and: 'file /tmp/pytest-of-fab/pytest-4/test_capture_with_fixture0/test_capture_with_fixture.py, line 3'
E           and: '  def test(subtests, capsys):'
E           and: "E       fixture 'subtests' not found"
E           and: '>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave'
E           and: ">       use 'pytest --fixtures [testpath]' for help on them."
E           and: ''
E           and: '/tmp/pytest-of-fab/pytest-4/test_capture_with_fixture0/test_capture_with_fixture.py:3'
E           and: '\r                                                                 \x1b[32m100% \x1b[0m\x1b[40m██████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.06s):'
E           and: '\x1b[31m       1 error\x1b[0m'
E           and: ''
E       remains unmatched: '*1 passed*'

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:308: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_capture_with_fixture0
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0


―――――――――――――――――――――――――――― ERROR at setup of test ――――――――――――――――――――――――――――
file /tmp/pytest-of-fab/pytest-4/test_capture_with_fixture0/test_capture_with_fixture.py, line 3
  def test(subtests, capsys):
E       fixture 'subtests' not found
>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-fab/pytest-4/test_capture_with_fixture0/test_capture_with_fixture.py:3
                                                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.06s):
       1 error

 tests/test_subtests.py::TestCapture.test_capture_with_fixture[capsys] ⨯                                                                                            95% █████████▌

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestCapture.test_capture_with_fixture[capfd] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestCapture object at 0x7f12c1372210>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-4/test_capture_with_fixture1')>, fixture = 'capfd'

    @pytest.mark.parametrize("fixture", ["capsys", "capfd"])
    def test_capture_with_fixture(self, testdir, fixture):
        testdir.makepyfile(
            r"""
            import sys
    
            def test(subtests, {fixture}):
                print('start test')
    
                with subtests.test(i='A'):
                    print("hello stdout A")
                    print("hello stderr A", file=sys.stderr)
    
                out, err = {fixture}.readouterr()
                assert out == 'start test\nhello stdout A\n'
                assert err == 'hello stderr A\n'
        """.format(
                fixture=fixture
            )
        )
        result = testdir.runpytest()
        result.stdout.fnmatch_lines(
>           ["*1 passed*",]
        )
E       Failed: nomatch: '*1 passed*'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-4/test_capture_with_fixture1'
E           and: 'plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: ''
E           and: ''
E           and: '―――――――――――――――――――――――――――― ERROR at setup of test ――――――――――――――――――――――――――――'
E           and: 'file /tmp/pytest-of-fab/pytest-4/test_capture_with_fixture1/test_capture_with_fixture.py, line 3'
E           and: '  def test(subtests, capfd):'
E           and: "E       fixture 'subtests' not found"
E           and: '>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave'
E           and: ">       use 'pytest --fixtures [testpath]' for help on them."
E           and: ''
E           and: '/tmp/pytest-of-fab/pytest-4/test_capture_with_fixture1/test_capture_with_fixture.py:3'
E           and: '\r                                                                 \x1b[32m100% \x1b[0m\x1b[40m██████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.04s):'
E           and: '\x1b[31m       1 error\x1b[0m'
E           and: ''
E       remains unmatched: '*1 passed*'

/home/fab/Documents/repos/pytest-subtests/tests/test_subtests.py:308: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/pytest-subtests/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-4/test_capture_with_fixture1
plugins: hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0


―――――――――――――――――――――――――――― ERROR at setup of test ――――――――――――――――――――――――――――
file /tmp/pytest-of-fab/pytest-4/test_capture_with_fixture1/test_capture_with_fixture.py, line 3
  def test(subtests, capfd):
E       fixture 'subtests' not found
>       available fixtures: _dj_autoclear_mailbox, _django_clear_site_cache, _django_db_marker, _django_set_urlconf, _django_setup_unittest, _fail_for_invalid_template_variable, _live_server_helper, _template_string_if_invalid_marker, _vcr_marker, admin_client, admin_user, autojump_clock, betamax_parametrized_recorder, betamax_parametrized_session, betamax_recorder, betamax_session, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, client, cov, datafiles, db, django_assert_max_num_queries, django_assert_num_queries, django_db_blocker, django_db_createdb, django_db_keepdb, django_db_modify_db_settings, django_db_modify_db_settings_parallel_suffix, django_db_modify_db_settings_tox_suffix, django_db_modify_db_settings_xdist_suffix, django_db_reset_sequences, django_db_setup, django_db_use_migrations, django_mail_dnsname, django_mail_patch_dns, django_test_environment, django_user_model, django_username_field, doctest_namespace, event_loop, live_server, loop, mailoutbox, mock, mock_clock, mocker, monkeypatch, no_cover, nursery, patching, print_logs, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, rf, settings, smart_caplog, stdouts, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, tmpworkdir, transactional_db, unused_tcp_port, unused_tcp_port_factory, vcr, vcr_cassette, vcr_cassette_dir, vcr_cassette_name, vcr_config, weave
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-fab/pytest-4/test_capture_with_fixture1/test_capture_with_fixture.py:3
                                                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.04s):
       1 error

 tests/test_subtests.py::TestCapture.test_capture_with_fixture[capfd] ⨯                                                                                            100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (6.78s):
       3 passed
       9 failed
         - tests/test_subtests.py:23 TestFixture.test_simple_terminal_normal[normal]
         - tests/test_subtests.py:39 TestFixture.test_simple_terminal_verbose[normal]
         - tests/test_subtests.py:71 TestFixture.test_skip[normal]
         - tests/test_subtests.py:116 TestSubTest.test_simple_terminal_normal[pytest-normal]
         - tests/test_subtests.py:150 TestSubTest.test_simple_terminal_verbose[pytest-normal]
         - tests/test_subtests.py:248 TestCapture.test_capturing
         - tests/test_subtests.py:270 TestCapture.test_no_capture
         - tests/test_subtests.py:286 TestCapture.test_capture_with_fixture[capsys]
         - tests/test_subtests.py:286 TestCapture.test_capture_with_fixture[capfd]
       2 xfailed
       5 skipped

At least it seems that a fixture is not found.

Tests for 0.3.0 are failing as well during the RPM build process for Fedora.

+ pytest-3.7 -v tests
/usr/lib/python3.7/site-packages/trio/_core/_multierror.py:450: RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing trio's custom handler, but this means MultiErrors will not show full tracebacks.
  category=RuntimeWarning
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')
rootdir: /home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0
plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0
collecting ... 

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestFixture.test_simple_terminal_normal[normal] ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestFixture object at 0x7f9f46c9c290>, simple_script = None, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-1/test_simple_terminal_normal0')>
mode = 'normal'

    def test_simple_terminal_normal(self, simple_script, testdir, mode):
        if mode == "normal":
            result = testdir.runpytest()
            expected_lines = ["collected 1 item"]
        else:
            pytest.importorskip("xdist")
            result = testdir.runpytest("-n1")
            expected_lines = ["gw0 [1]"]
    
        expected_lines += [
            "* test_foo [[]custom[]] (i=1) *",
            "* test_foo [[]custom[]] (i=3) *",
            "* 2 failed, 1 passed in *",
        ]
>       result.stdout.fnmatch_lines(expected_lines)
E       Failed: nomatch: 'collected 1 item'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-1/test_simple_terminal_normal0'
E           and: 'plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: '\r'
E           and: ''
E           and: ''
E           and: '――――――――――――――――――――――――――― test_foo [custom] (i=1) ――――――――――――――――――――――――――――'
E           and: ''
E           and: "subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f46ea42d0>, suspend_capture_ctx=<bound method CaptureManager.gl... _in_suspended='<UNSET>'> _current_item=<Function test_foo>>>, request=<SubRequest 'subtests' for <Function test_foo>>)"
E           and: ''
E           and: '    def test_foo(subtests):'
E           and: '        for i in range(5):'
E           and: '            with subtests.test(msg="custom", i=i):'
E           and: '>               assert i % 2 == 0'
E           and: 'E               assert (1 % 2) == 0'
E           and: ''
E           and: 'test_simple_terminal_normal.py:4: AssertionError'
E           and: '\r'
E           and: ''
E           and: ''
E           and: '――――――――――――――――――――――――――― test_foo [custom] (i=3) ――――――――――――――――――――――――――――'
E           and: ''
E           and: "subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f46ea42d0>, suspend_capture_ctx=<bound method CaptureManager.gl... _in_suspended='<UNSET>'> _current_item=<Function test_foo>>>, request=<SubRequest 'subtests' for <Function test_foo>>)"
E           and: ''
E           and: '    def test_foo(subtests):'
E           and: '        for i in range(5):'
E           and: '            with subtests.test(msg="custom", i=i):'
E           and: '>               assert i % 2 == 0'
E           and: 'E               assert (3 % 2) == 0'
E           and: ''
E           and: 'test_simple_terminal_normal.py:4: AssertionError'
E           and: '\r'
E           and: '\r \x1b[36m\x1b[0mtest_simple_terminal_normal.py\x1b[0m \x1b[31m⨯\x1b[0m\x1b[32m✓\x1b[0m\x1b[32m✓\x1b[0m                              \x1b[31m100% \x1b[0m\x1b[40m\x1b[31m█\x1b[0m\x1b[40m\x1b[31m█████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.13s):'
E           and: '\x1b[32m       4 passed\x1b[0m'
E           and: '\x1b[31m       2 failed\x1b[0m'
E           and: '         - \x1b[36m\x1b[0mtest_simple_terminal_normal.py\x1b[0m:1 \x1b[31mtest_foo\x1b[0m'
E           and: '         - \x1b[36m\x1b[0mtest_simple_terminal_normal.py\x1b[0m:1 \x1b[31mtest_foo\x1b[0m'
E           and: ''
E       remains unmatched: 'collected 1 item'

/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/tests/test_subtests.py:37: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-1/test_simple_terminal_normal0
plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0



[...]
rootdir: /tmp/pytest-of-fab/pytest-1/test_simple_terminal_normal3
plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0

―――――――――――――――――――――――――― T.test_foo [custom] (i=1) ―――――――――――――――――――――――――――

self = <test_simple_terminal_normal.T testMethod=test_foo>

    def test_foo(self):
        for i in range(5):
            with self.subTest(msg="custom", i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_simple_terminal_normal.py:8: AssertionError


―――――――――――――――――――――――――― T.test_foo [custom] (i=3) ―――――――――――――――――――――――――――

self = <test_simple_terminal_normal.T testMethod=test_foo>

    def test_foo(self):
        for i in range(5):
            with self.subTest(msg="custom", i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_simple_terminal_normal.py:8: AssertionError

 test_simple_terminal_normal.py ⨯✓                               100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.07s):
       1 passed
       2 failed
         - test_simple_terminal_normal.py:5 T.test_foo
         - test_simple_terminal_normal.py:5 T.test_foo

 tests/test_subtests.py::TestSubTest.test_simple_terminal_normal[pytest-normal] ⨯                                                                                   42% ████▎     
 tests/test_subtests.py::TestSubTest.test_simple_terminal_normal[pytest-xdist] s                                                                                    47% ████▊     
 tests/test_subtests.py::TestSubTest.test_simple_terminal_verbose[unittest] ✓                                                                                       53% █████▍    

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestSubTest.test_simple_terminal_verbose[pytest-normal] ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestSubTest object at 0x7f9f532ab3d0>, simple_script = local('/tmp/pytest-of-fab/pytest-1/test_simple_terminal_verbose3/test_simple_terminal_verbose.py')
testdir = <Testdir local('/tmp/pytest-of-fab/pytest-1/test_simple_terminal_verbose3')>, runner = 'pytest-normal'

    @pytest.mark.parametrize("runner", ["unittest", "pytest-normal", "pytest-xdist"])
    def test_simple_terminal_verbose(self, simple_script, testdir, runner):
    
        if runner == "unittest":
            result = testdir.run(sys.executable, simple_script, "-v")
            result.stderr.fnmatch_lines(
                [
                    "test_foo (__main__.T) ... ",
                    "FAIL: test_foo (__main__.T) [custom] (i=1)",
                    "AssertionError: 1 != 0",
                    "FAIL: test_foo (__main__.T) [custom] (i=3)",
                    "AssertionError: 1 != 0",
                    "Ran 1 test in *",
                    "FAILED (failures=2)",
                ]
            )
        else:
            if runner == "pytest-normal":
                result = testdir.runpytest(simple_script, "-v")
                expected_lines = [
                    "*collected 1 item",
                    "test_simple_terminal_verbose.py::T::test_foo FAILED *100%*",
                    "test_simple_terminal_verbose.py::T::test_foo FAILED *100%*",
                    "test_simple_terminal_verbose.py::T::test_foo PASSED *100%*",
                ]
            else:
                pytest.importorskip("xdist")
                result = testdir.runpytest(simple_script, "-n1", "-v")
                expected_lines = [
                    "gw0 [1]",
                    "*gw0*100%* FAILED test_simple_terminal_verbose.py::T::test_foo*",
                    "*gw0*100%* FAILED test_simple_terminal_verbose.py::T::test_foo*",
                    "*gw0*100%* PASSED test_simple_terminal_verbose.py::T::test_foo*",
                ]
            result.stdout.fnmatch_lines(
                expected_lines
                + [
                    "* T.test_foo [[]custom[]] (i=1) *",
                    "E  * AssertionError: 1 != 0",
                    "* T.test_foo [[]custom[]] (i=3) *",
                    "E  * AssertionError: 1 != 0",
>                   "* 2 failed, 1 passed in *",
                ]
            )
E           Failed: nomatch: '*collected 1 item'
E               and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E               and: 'cachedir: .pytest_cache'
E               and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')"
E               and: 'rootdir: /tmp/pytest-of-fab/pytest-1/test_simple_terminal_verbose3'
E               and: 'plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E               and: 'collecting ... '
E               and: '―――――――――――――――――――――――――― T.test_foo [custom] (i=1) ―――――――――――――――――――――――――――'
E               and: ''
E               and: 'self = <test_simple_terminal_verbose.T testMethod=test_foo>'
E               and: ''
E               and: '    def test_foo(self):'
E               and: '        for i in range(5):'
E               and: '            with self.subTest(msg="custom", i=i):'
E               and: '>               self.assertEqual(i % 2, 0)'
E               and: 'E               AssertionError: 1 != 0'
E               and: ''
E               and: 'test_simple_terminal_verbose.py:8: AssertionError'
E               and: '\r'
E               and: ''
E               and: '―――――――――――――――――――――――――― T.test_foo [custom] (i=3) ―――――――――――――――――――――――――――'
E               and: ''
E               and: 'self = <test_simple_terminal_verbose.T testMethod=test_foo>'
E               and: ''
E               and: '    def test_foo(self):'
E               and: '        for i in range(5):'
E               and: '            with self.subTest(msg="custom", i=i):'
E               and: '>               self.assertEqual(i % 2, 0)'
E               and: 'E               AssertionError: 1 != 0'
E               and: ''
E               and: 'test_simple_terminal_verbose.py:8: AssertionError'
E               and: '\r'
E               and: '\r \x1b[36mtest_simple_terminal_verbose.py\x1b[0m::T.test_foo\x1b[0m \x1b[31m⨯\x1b[0m\x1b[32m✓\x1b[0m                  \x1b[31m100% \x1b[0m\x1b[40m\x1b[31m█\x1b[0m\x1b[40m\x1b[31m█████████\x1b[0m'
E               and: '===Flaky Test Report==='
E               and: ''
E               and: ''
E               and: '===End Flaky Test Report==='
E               and: ''
E               and: 'Results (0.06s):'
E               and: '\x1b[32m       1 passed\x1b[0m'
E               and: '\x1b[31m       2 failed\x1b[0m'
E               and: '         - \x1b[36m\x1b[0mtest_simple_terminal_verbose.py\x1b[0m:5 \x1b[31mT.test_foo\x1b[0m'
E               and: '         - \x1b[36m\x1b[0mtest_simple_terminal_verbose.py\x1b[0m:5 \x1b[31mT.test_foo\x1b[0m'
E               and: ''
E           remains unmatched: '*collected 1 item'

/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/tests/test_subtests.py:191: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-1/test_simple_terminal_verbose3
plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0
collecting ... 
―――――――――――――――――――――――――― T.test_foo [custom] (i=1) ―――――――――――――――――――――――――――

self = <test_simple_terminal_verbose.T testMethod=test_foo>

    def test_foo(self):
        for i in range(5):
            with self.subTest(msg="custom", i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_simple_terminal_verbose.py:8: AssertionError


―――――――――――――――――――――――――― T.test_foo [custom] (i=3) ―――――――――――――――――――――――――――

self = <test_simple_terminal_verbose.T testMethod=test_foo>

    def test_foo(self):
        for i in range(5):
            with self.subTest(msg="custom", i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_simple_terminal_verbose.py:8: AssertionError

 test_simple_terminal_verbose.py::T.test_foo ⨯✓                  100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.06s):
       1 passed
       2 failed
         - test_simple_terminal_verbose.py:5 T.test_foo
         - test_simple_terminal_verbose.py:5 T.test_foo

 tests/test_subtests.py::TestSubTest.test_simple_terminal_verbose[pytest-normal] ⨯                                                                                  58% █████▊    
 tests/test_subtests.py::TestSubTest.test_simple_terminal_verbose[pytest-xdist] s                                                                                   63% ██████▍   
 tests/test_subtests.py::TestSubTest.test_skip[unittest] ✓                                                                                                          68% ██████▉   
 tests/test_subtests.py::TestSubTest.test_skip[pytest-normal] x                                                                                                     74% ███████▍  
 tests/test_subtests.py::TestSubTest.test_skip[pytest-xdist] x                                                                                                      79% ███████▉  

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestCapture.test_capturing ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestCapture object at 0x7f9f468f91d0>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-1/test_capturing0')>

    def test_capturing(self, testdir):
        self.create_file(testdir)
        result = testdir.runpytest()
        result.stdout.fnmatch_lines(
            [
                "*__ test (i='A') __*",
                "*Captured stdout call*",
                "hello stdout A",
                "*Captured stderr call*",
                "hello stderr A",
                "*__ test (i='B') __*",
                "*Captured stdout call*",
                "hello stdout B",
                "*Captured stderr call*",
                "hello stderr B",
                "*__ test __*",
                "*Captured stdout call*",
                "start test",
>               "end test",
            ]
        )
E       Failed: nomatch: "*__ test (i='A') __*"
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-1/test_capturing0'
E           and: 'plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: ''
E           and: ''
E           and: "――――――――――――――――――――――――――――――――― test (i='A') ―――――――――――――――――――――――――――――――――"
E           and: ''
E           and: "subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f4694c550>, suspend_capture_ctx=<bound method CaptureManager.gl...resumed' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)"
E           and: ''
E           and: '    def test(subtests):'
E           and: '        print()'
E           and: "        print('start test')"
E           and: '    '
E           and: "        with subtests.test(i='A'):"
E           and: '            print("hello stdout A")'
E           and: '            print("hello stderr A", file=sys.stderr)'
E           and: '>           assert 0'
E           and: 'E           assert 0'
E           and: ''
E           and: 'test_capturing.py:9: AssertionError'
E           and: '----------------------------- Captured stdout call -----------------------------'
E           and: 'hello stdout A'
E           and: '----------------------------- Captured stderr call -----------------------------'
E           and: 'hello stderr A'
E           and: '\r'
E           and: ''
E           and: ''
E           and: "――――――――――――――――――――――――――――――――― test (i='B') ―――――――――――――――――――――――――――――――――"
E           and: ''
E           and: "subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f4694c550>, suspend_capture_ctx=<bound method CaptureManager.gl...resumed' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)"
E           and: ''
E           and: '    def test(subtests):'
E           and: '        print()'
E           and: "        print('start test')"
E           and: '    '
E           and: "        with subtests.test(i='A'):"
E           and: '            print("hello stdout A")'
E           and: '            print("hello stderr A", file=sys.stderr)'
E           and: '            assert 0'
E           and: '    '
E           and: "        with subtests.test(i='B'):"
E           and: '            print("hello stdout B")'
E           and: '            print("hello stderr B", file=sys.stderr)'
E           and: '>           assert 0'
E           and: 'E           assert 0'
E           and: ''
E           and: 'test_capturing.py:14: AssertionError'
E           and: '----------------------------- Captured stdout call -----------------------------'
E           and: 'hello stdout B'
E           and: '----------------------------- Captured stderr call -----------------------------'
E           and: 'hello stderr B'
E           and: '\r'
E           and: ''
E           and: ''
E           and: '――――――――――――――――――――――――――――――――――――― test ―――――――――――――――――――――――――――――――――――――'
E           and: ''
E           and: "subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f4694c550>, suspend_capture_ctx=<bound method CaptureManager.gl...spended' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)"
E           and: ''
E           and: '    def test(subtests):'
E           and: '        print()'
E           and: "        print('start test')"
E           and: '    '
E           and: "        with subtests.test(i='A'):"
E           and: '            print("hello stdout A")'
E           and: '            print("hello stderr A", file=sys.stderr)'
E           and: '            assert 0'
E           and: '    '
E           and: "        with subtests.test(i='B'):"
E           and: '            print("hello stdout B")'
E           and: '            print("hello stderr B", file=sys.stderr)'
E           and: '            assert 0'
E           and: '    '
E           and: "        print('end test')"
E           and: '>       assert 0'
E           and: 'E       assert 0'
E           and: ''
E           and: 'test_capturing.py:17: AssertionError'
E           and: '----------------------------- Captured stdout call -----------------------------'
E           and: ''
E           and: 'start test'
E           and: 'end test'
E           and: '\r'
E           and: '\r \x1b[36m\x1b[0mtest_capturing.py\x1b[0m \x1b[31m⨯\x1b[0m                                             \x1b[31m100% \x1b[0m\x1b[40m\x1b[31m█\x1b[0m\x1b[40m\x1b[31m█████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.10s):'
E           and: '\x1b[31m       3 failed\x1b[0m'
E           and: '         - \x1b[36m\x1b[0mtest_capturing.py\x1b[0m:2 \x1b[31mtest\x1b[0m'
E           and: '         - \x1b[36m\x1b[0mtest_capturing.py\x1b[0m:2 \x1b[31mtest\x1b[0m'
E           and: '         - \x1b[36m\x1b[0mtest_capturing.py\x1b[0m:2 \x1b[31mtest\x1b[0m'
E           and: ''
E       remains unmatched: "*__ test (i='A') __*"

/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/tests/test_subtests.py:266: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-1/test_capturing0
plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0


――――――――――――――――――――――――――――――――― test (i='A') ―――――――――――――――――――――――――――――――――

subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f4694c550>, suspend_capture_ctx=<bound method CaptureManager.gl...resumed' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)

    def test(subtests):
        print()
        print('start test')
    
        with subtests.test(i='A'):
            print("hello stdout A")
            print("hello stderr A", file=sys.stderr)
>           assert 0
E           assert 0

test_capturing.py:9: AssertionError
----------------------------- Captured stdout call -----------------------------
hello stdout A
----------------------------- Captured stderr call -----------------------------
hello stderr A



――――――――――――――――――――――――――――――――― test (i='B') ―――――――――――――――――――――――――――――――――

subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f4694c550>, suspend_capture_ctx=<bound method CaptureManager.gl...resumed' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)

    def test(subtests):
        print()
        print('start test')
    
        with subtests.test(i='A'):
            print("hello stdout A")
            print("hello stderr A", file=sys.stderr)
            assert 0
    
        with subtests.test(i='B'):
            print("hello stdout B")
            print("hello stderr B", file=sys.stderr)
>           assert 0
E           assert 0

test_capturing.py:14: AssertionError
----------------------------- Captured stdout call -----------------------------
hello stdout B
----------------------------- Captured stderr call -----------------------------
hello stderr B



――――――――――――――――――――――――――――――――――――― test ―――――――――――――――――――――――――――――――――――――

subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f4694c550>, suspend_capture_ctx=<bound method CaptureManager.gl...spended' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)

    def test(subtests):
        print()
        print('start test')
    
        with subtests.test(i='A'):
            print("hello stdout A")
            print("hello stderr A", file=sys.stderr)
            assert 0
    
        with subtests.test(i='B'):
            print("hello stdout B")
            print("hello stderr B", file=sys.stderr)
            assert 0
    
        print('end test')
>       assert 0
E       assert 0

test_capturing.py:17: AssertionError
----------------------------- Captured stdout call -----------------------------

start test
end test

 test_capturing.py ⨯                                             100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.10s):
       3 failed
         - test_capturing.py:2 test
         - test_capturing.py:2 test
         - test_capturing.py:2 test

 tests/test_subtests.py::TestCapture.test_capturing ⨯                                                                                                               84% ████████▌ 

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestCapture.test_no_capture ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestCapture object at 0x7f9f4691e690>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-1/test_no_capture0')>

    def test_no_capture(self, testdir):
        self.create_file(testdir)
        result = testdir.runpytest("-s")
        result.stdout.fnmatch_lines(
            [
                "start test",
                "hello stdout A",
                "Fhello stdout B",
                "Fend test",
                "*__ test (i='A') __*",
                "*__ test (i='B') __*",
>               "*__ test __*",
            ]
        )
E       Failed: nomatch: 'start test'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-1/test_no_capture0'
E           and: 'plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: ''
E       exact match: 'start test'
E       exact match: 'hello stdout A'
E       nomatch: 'Fhello stdout B'
E           and: ''
E           and: ''
E           and: "――――――――――――――――――――――――――――――――― test (i='A') ―――――――――――――――――――――――――――――――――"
E           and: ''
E           and: "subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f46838b90>, suspend_capture_ctx=<bound method CaptureManager.gl...resumed' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)"
E           and: ''
E           and: '    def test(subtests):'
E           and: '        print()'
E           and: "        print('start test')"
E           and: '    '
E           and: "        with subtests.test(i='A'):"
E           and: '            print("hello stdout A")'
E           and: '            print("hello stderr A", file=sys.stderr)'
E           and: '>           assert 0'
E           and: 'E           assert 0'
E           and: ''
E           and: 'test_no_capture.py:9: AssertionError'
E           and: '\r'
E           and: 'hello stdout B'
E           and: ''
E           and: ''
E           and: "――――――――――――――――――――――――――――――――― test (i='B') ―――――――――――――――――――――――――――――――――"
E           and: ''
E           and: "subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f46838b90>, suspend_capture_ctx=<bound method CaptureManager.gl...resumed' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)"
E           and: ''
E           and: '    def test(subtests):'
E           and: '        print()'
E           and: "        print('start test')"
E           and: '    '
E           and: "        with subtests.test(i='A'):"
E           and: '            print("hello stdout A")'
E           and: '            print("hello stderr A", file=sys.stderr)'
E           and: '            assert 0'
E           and: '    '
E           and: "        with subtests.test(i='B'):"
E           and: '            print("hello stdout B")'
E           and: '            print("hello stderr B", file=sys.stderr)'
E           and: '>           assert 0'
E           and: 'E           assert 0'
E           and: ''
E           and: 'test_no_capture.py:14: AssertionError'
E           and: '\r'
E           and: 'end test'
E           and: ''
E           and: ''
E           and: '――――――――――――――――――――――――――――――――――――― test ―――――――――――――――――――――――――――――――――――――'
E           and: ''
E           and: "subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f46838b90>, suspend_capture_ctx=<bound method CaptureManager.gl...spended' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)"
E           and: ''
E           and: '    def test(subtests):'
E           and: '        print()'
E           and: "        print('start test')"
E           and: '    '
E           and: "        with subtests.test(i='A'):"
E           and: '            print("hello stdout A")'
E           and: '            print("hello stderr A", file=sys.stderr)'
E           and: '            assert 0'
E           and: '    '
E           and: "        with subtests.test(i='B'):"
E           and: '            print("hello stdout B")'
E           and: '            print("hello stderr B", file=sys.stderr)'
E           and: '            assert 0'
E           and: '    '
E           and: "        print('end test')"
E           and: '>       assert 0'
E           and: 'E       assert 0'
E           and: ''
E           and: 'test_no_capture.py:17: AssertionError'
E           and: '\r'
E           and: '\r \x1b[36m\x1b[0mtest_no_capture.py\x1b[0m \x1b[31m⨯\x1b[0m                                            \x1b[31m100% \x1b[0m\x1b[40m\x1b[31m█\x1b[0m\x1b[40m\x1b[31m█████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.06s):'
E           and: '\x1b[31m       3 failed\x1b[0m'
E           and: '         - \x1b[36m\x1b[0mtest_no_capture.py\x1b[0m:2 \x1b[31mtest\x1b[0m'
E           and: '         - \x1b[36m\x1b[0mtest_no_capture.py\x1b[0m:2 \x1b[31mtest\x1b[0m'
E           and: '         - \x1b[36m\x1b[0mtest_no_capture.py\x1b[0m:2 \x1b[31mtest\x1b[0m'
E           and: ''
E       remains unmatched: 'Fhello stdout B'

/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/tests/test_subtests.py:281: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-1/test_no_capture0
plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0

start test
hello stdout A


――――――――――――――――――――――――――――――――― test (i='A') ―――――――――――――――――――――――――――――――――

subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f46838b90>, suspend_capture_ctx=<bound method CaptureManager.gl...resumed' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)

    def test(subtests):
        print()
        print('start test')
    
        with subtests.test(i='A'):
            print("hello stdout A")
            print("hello stderr A", file=sys.stderr)
>           assert 0
E           assert 0

test_no_capture.py:9: AssertionError

hello stdout B


――――――――――――――――――――――――――――――――― test (i='B') ―――――――――――――――――――――――――――――――――

subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f46838b90>, suspend_capture_ctx=<bound method CaptureManager.gl...resumed' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)

    def test(subtests):
        print()
        print('start test')
    
        with subtests.test(i='A'):
            print("hello stdout A")
            print("hello stderr A", file=sys.stderr)
            assert 0
    
        with subtests.test(i='B'):
            print("hello stdout B")
            print("hello stderr B", file=sys.stderr)
>           assert 0
E           assert 0

test_no_capture.py:14: AssertionError

end test


――――――――――――――――――――――――――――――――――――― test ―――――――――――――――――――――――――――――――――――――

subtests = SubTests(ihook=<pluggy.hooks._HookRelay object at 0x7f9f46838b90>, suspend_capture_ctx=<bound method CaptureManager.gl...spended' _in_suspended='<UNSET>'> _current_item=<Function test>>>, request=<SubRequest 'subtests' for <Function test>>)

    def test(subtests):
        print()
        print('start test')
    
        with subtests.test(i='A'):
            print("hello stdout A")
            print("hello stderr A", file=sys.stderr)
            assert 0
    
        with subtests.test(i='B'):
            print("hello stdout B")
            print("hello stderr B", file=sys.stderr)
            assert 0
    
        print('end test')
>       assert 0
E       assert 0

test_no_capture.py:17: AssertionError

 test_no_capture.py ⨯                                            100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.06s):
       3 failed
         - test_no_capture.py:2 test
         - test_no_capture.py:2 test
         - test_no_capture.py:2 test
------------------------------------------------------------------------------ Captured stderr call ------------------------------------------------------------------------------
hello stderr A
hello stderr B

 tests/test_subtests.py::TestCapture.test_no_capture ⨯                                                                                                              89% ████████▉ 

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestCapture.test_capture_with_fixture[capsys] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestCapture object at 0x7f9f4677c3d0>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-1/test_capture_with_fixture0')>, fixture = 'capsys'

    @pytest.mark.parametrize("fixture", ["capsys", "capfd"])
    def test_capture_with_fixture(self, testdir, fixture):
        testdir.makepyfile(
            r"""
            import sys
    
            def test(subtests, {fixture}):
                print('start test')
    
                with subtests.test(i='A'):
                    print("hello stdout A")
                    print("hello stderr A", file=sys.stderr)
    
                out, err = {fixture}.readouterr()
                assert out == 'start test\nhello stdout A\n'
                assert err == 'hello stderr A\n'
        """.format(
                fixture=fixture
            )
        )
        result = testdir.runpytest()
        result.stdout.fnmatch_lines(
>           ["*1 passed*",]
        )
E       Failed: nomatch: '*1 passed*'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-1/test_capture_with_fixture0'
E           and: 'plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: '\r'
E           and: '\r \x1b[36m\x1b[0mtest_capture_with_fixture.py\x1b[0m \x1b[32m✓\x1b[0m\x1b[32m✓\x1b[0m                                 \x1b[32m100% \x1b[0m\x1b[40m\x1b[32m█\x1b[0m\x1b[40m\x1b[32m█████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.05s):'
E           and: '\x1b[32m       2 passed\x1b[0m'
E           and: ''
E       remains unmatched: '*1 passed*'

/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/tests/test_subtests.py:308: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-1/test_capture_with_fixture0
plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0

 test_capture_with_fixture.py ✓✓                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.05s):
       2 passed

 tests/test_subtests.py::TestCapture.test_capture_with_fixture[capsys] ⨯                                                                                            95% █████████▌

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestCapture.test_capture_with_fixture[capfd] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

self = <test_subtests.TestCapture object at 0x7f9f46788610>, testdir = <Testdir local('/tmp/pytest-of-fab/pytest-1/test_capture_with_fixture1')>, fixture = 'capfd'

    @pytest.mark.parametrize("fixture", ["capsys", "capfd"])
    def test_capture_with_fixture(self, testdir, fixture):
        testdir.makepyfile(
            r"""
            import sys
    
            def test(subtests, {fixture}):
                print('start test')
    
                with subtests.test(i='A'):
                    print("hello stdout A")
                    print("hello stderr A", file=sys.stderr)
    
                out, err = {fixture}.readouterr()
                assert out == 'start test\nhello stdout A\n'
                assert err == 'hello stderr A\n'
        """.format(
                fixture=fixture
            )
        )
        result = testdir.runpytest()
        result.stdout.fnmatch_lines(
>           ["*1 passed*",]
        )
E       Failed: nomatch: '*1 passed*'
E           and: 'Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')"
E           and: 'rootdir: /tmp/pytest-of-fab/pytest-1/test_capture_with_fixture1'
E           and: 'plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0'
E           and: '\r'
E           and: '\r \x1b[36m\x1b[0mtest_capture_with_fixture.py\x1b[0m \x1b[32m✓\x1b[0m\x1b[32m✓\x1b[0m                                 \x1b[32m100% \x1b[0m\x1b[40m\x1b[32m█\x1b[0m\x1b[40m\x1b[32m█████████\x1b[0m'
E           and: '===Flaky Test Report==='
E           and: ''
E           and: ''
E           and: '===End Flaky Test Report==='
E           and: ''
E           and: 'Results (0.05s):'
E           and: '\x1b[32m       2 passed\x1b[0m'
E           and: ''
E       remains unmatched: '*1 passed*'

/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/tests/test_subtests.py:308: Failed
------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------
Test session starts (platform: linux, Python 3.7.6, pytest 4.6.9, pytest-sugar 0.9.2)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/fab/Documents/repos/rpmbuild/BUILD/pytest-subtests-0.3.0/.hypothesis/examples')
rootdir: /tmp/pytest-of-fab/pytest-1/test_capture_with_fixture1
plugins: subtests-0.3.0, hypothesis-4.23.8, requests-mock-1.7.0, case-1.5.3, sugar-0.9.2, betamax-0.8.1, asyncio-0.10.0, toolbox-0.4, timeout-1.3.3, cov-2.8.1, isort-0.3.1, forked-1.0.2, datafiles-2.0, vcr-1.0.2, aspectlib-1.4.2, mock-1.10.4, trio-0.5.2, flaky-3.5.3, django-3.7.0

 test_capture_with_fixture.py ✓✓                                 100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (0.05s):
       2 passed

 tests/test_subtests.py::TestCapture.test_capture_with_fixture[capfd] ⨯                                                                                            100% ██████████
===Flaky Test Report===


===End Flaky Test Report===

Results (5.99s):
       3 passed
       9 failed
         - tests/test_subtests.py:23 TestFixture.test_simple_terminal_normal[normal]
         - tests/test_subtests.py:39 TestFixture.test_simple_terminal_verbose[normal]
         - tests/test_subtests.py:71 TestFixture.test_skip[normal]
         - tests/test_subtests.py:116 TestSubTest.test_simple_terminal_normal[pytest-normal]
         - tests/test_subtests.py:150 TestSubTest.test_simple_terminal_verbose[pytest-normal]
         - tests/test_subtests.py:248 TestCapture.test_capturing
         - tests/test_subtests.py:270 TestCapture.test_no_capture
         - tests/test_subtests.py:286 TestCapture.test_capture_with_fixture[capsys]
         - tests/test_subtests.py:286 TestCapture.test_capture_with_fixture[capfd]
       2 xfailed
       5 skipped
error: Bad exit status from /var/tmp/rpm-tmp.U5gasx (%check)
@Alessandro-Barbieri
Copy link

Same while trying to package this for gentoo

@nicoddemus
Copy link
Member

Hey @fabaff and @Alessandro-Barbieri, sorry for the delay.

The subtests fixture is provided by the plugin itself, so it definitely should have been found.

After activating the environment and executing pytest setup.py develop, are you able to execute a simple example like this?

def test(subtests):
    for i in range(5):
        with subtests.test(msg="custom message", i=i):
            assert i % 2 == 0

@Alessandro-Barbieri which pytest version are you using? I see @fabaff is using 4.6.9.

Alessandro-Barbieri referenced this issue in gentoo/guru Apr 3, 2020
@AndrewAmmerlaan
dev-python/pytest-subtests: fix tests if pakcage ...
0e4e63d 
is not already installed

Package-Manager: Portage-2.3.96, Repoman-2.3.22
Signed-off-by: Andrew Ammerlaan <andrewammerlaan@riseup.net>
@Alessandro-Barbieri
Copy link

Solved with a PYTHONPATH workaround

@nicoddemus
Copy link
Member

@Alessandro-Barbieri were you installing it in a virtualenv?

@Alessandro-Barbieri
Copy link

@nicoddemus well ... I don't know what the gentoo eclass is really doing behind the scenes

@nicoddemus
Copy link
Member

Sorry, not sure how else I can help here... I'm closing for now, feel free to post more information so we can reopen this then. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@fabaff @nicoddemus @Alessandro-Barbieri