Python testing using new EOT token PYTEST - test output

[lszomoru]

Testing #21092

Verify that test run and logging function correctly by referring to test_logging.py. Ensure that ALL the expected output gets printed. Compare the output in VSCode with what you get when running pytest directly from the command line for the test_logging file.

There is a chance that I misunderstood the steps here (test_logging.py does not exist in the project) but I see some differences in logging when running the test in VS Code compared to just running pytest in the integrated terminal:

• Separators between the tests
• Short test summary info at the end

VS Code

/Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite/test_many_functions.py::test_many_2f failed: def test_many_2f():
>       assert inc_dec.increment(3) == -1
E       assert 4 == -1
E        +  where 4 = <function increment at 0x1055269d0>(3)
E        +    where <function increment at 0x1055269d0> = inc_dec.increment

test_many_functions.py:12: AssertionError


/Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite/test_many_functions.py::test_many_3f failed: def test_many_3f():
>       assert inc_dec.increment(-1) == 2
E       assert 0 == 2
E        +  where 0 = <function increment at 0x1055269d0>(-1)
E        +    where <function increment at 0x1055269d0> = inc_dec.increment

test_many_functions.py:16: AssertionError


/Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite/test_single_function.py::test_single_1f failed: def test_single_1f():
>       assert inc_dec.increment(-1) == 2
E       assert 0 == 2
E        +  where 0 = <function increment at 0x1055269d0>(-1)
E        +    where <function increment at 0x1055269d0> = inc_dec.increment

test_single_function.py:4: AssertionError


/Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite/test_subtests.py::NumbersTest::test_even failed: self = <test_subtests.NumbersTest testMethod=test_even>

    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 10000):
            with self.subTest(i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_subtests.py:11: AssertionError


/Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite/test_two_classes.py::ClassA::test_a2 failed: self = <test_two_classes.ClassA testMethod=test_a2>

    def test_a2(self):
        time.sleep(1)
>       self.assertEqual(inc_dec.decrement(7), 3, "Should be 6")
E       AssertionError: 6 != 3 : Should be 6

test_two_classes.py:13: AssertionError


/Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite/test_two_classes.py::ClassB::test_b2 failed: self = <test_two_classes.ClassB testMethod=test_b2>

    def test_b2(self):
>       self.assertEqual(inc_dec.decrement(7), 3, "Should be 6")
E       AssertionError: 6 != 3 : Should be 6

test_two_classes.py:25: AssertionError


/Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite/test_unit_and_pytest_combo.py::test_single_pytest failed: def test_single_pytest():
>       assert inc_dec.increment(-1) == 2
E       assert 0 == 2
E        +  where 0 = <function increment at 0x1055269d0>(-1)
E        +    where <function increment at 0x1055269d0> = inc_dec.increment

test_unit_and_pytest_combo.py:6: AssertionError


/Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite/outer_nested_folder/inner_nested_folder/test_unittest.py::test_class_unittest::test2_function_unittest failed: self = <test_unittest.test_class_unittest testMethod=test2_function_unittest>

    def test2_function_unittest(self):
>       self.assertEqual(inc_dec.decrement(7), 3, "Should be 6")
E       AssertionError: 6 != 3 : Should be 6

outer_nested_folder/inner_nested_folder/test_unittest.py:10: AssertionError


Finished running tests!

Pytest

================================================================================================================= test session starts =================================================================================================================
platform darwin -- Python 3.9.6, pytest-7.4.2, pluggy-1.3.0
rootdir: /Users/lszomoru/Source/other/inc_dec_example_repo/inc_dec_example_test_suite
collected 35 items                                                                                                                                                                                                                                    

diff_pattern_test.py .                                                                                                                                                                                                                          [  2%]
first_test.py .                                                                                                                                                                                                                                 [  5%]
test_docstring.txt .                                                                                                                                                                                                                            [  8%]
test_many_functions.py .FF                                                                                                                                                                                                                      [ 17%]
test_parameterized.py .......                                                                                                                                                                                                                   [ 37%]
test_single_function.py F                                                                                                                                                                                                                       [ 40%]
test_subtests.py F                                                                                                                                                                                                                              [ 42%]
test_two_classes.py .F.F                                                                                                                                                                                                                        [ 54%]
test_unit_and_pytest_combo.py F.                                                                                                                                                                                                                [ 60%]
test_wait_parameterized.py .........                                                                                                                                                                                                            [ 85%]
outer_nested_folder/test_outer_folder.py ..                                                                                                                                                                                                     [ 91%]
outer_nested_folder/inner_nested_folder/test_nested.py .                                                                                                                                                                                        [ 94%]
outer_nested_folder/inner_nested_folder/test_unittest.py .F                                                                                                                                                                                     [100%]

====================================================================================================================== FAILURES =======================================================================================================================
____________________________________________________________________________________________________________________ test_many_2f _____________________________________________________________________________________________________________________

    def test_many_2f():
>       assert inc_dec.increment(3) == -1
E       assert 4 == -1
E        +  where 4 = <function increment at 0x101512dc0>(3)
E        +    where <function increment at 0x101512dc0> = inc_dec.increment

test_many_functions.py:12: AssertionError
____________________________________________________________________________________________________________________ test_many_3f _____________________________________________________________________________________________________________________

    def test_many_3f():
>       assert inc_dec.increment(-1) == 2
E       assert 0 == 2
E        +  where 0 = <function increment at 0x101512dc0>(-1)
E        +    where <function increment at 0x101512dc0> = inc_dec.increment

test_many_functions.py:16: AssertionError
___________________________________________________________________________________________________________________ test_single_1f ____________________________________________________________________________________________________________________

    def test_single_1f():
>       assert inc_dec.increment(-1) == 2
E       assert 0 == 2
E        +  where 0 = <function increment at 0x101512dc0>(-1)
E        +    where <function increment at 0x101512dc0> = inc_dec.increment

test_single_function.py:4: AssertionError
________________________________________________________________________________________________________________ NumbersTest.test_even ________________________________________________________________________________________________________________

self = <test_subtests.NumbersTest testMethod=test_even>

    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 10000):
            with self.subTest(i=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_subtests.py:11: AssertionError
___________________________________________________________________________________________________________________ ClassA.test_a2 ____________________________________________________________________________________________________________________

self = <test_two_classes.ClassA testMethod=test_a2>

    def test_a2(self):
        time.sleep(1)
>       self.assertEqual(inc_dec.decrement(7), 3, "Should be 6")
E       AssertionError: 6 != 3 : Should be 6

test_two_classes.py:13: AssertionError
___________________________________________________________________________________________________________________ ClassB.test_b2 ____________________________________________________________________________________________________________________

self = <test_two_classes.ClassB testMethod=test_b2>

    def test_b2(self):
>       self.assertEqual(inc_dec.decrement(7), 3, "Should be 6")
E       AssertionError: 6 != 3 : Should be 6

test_two_classes.py:25: AssertionError
_________________________________________________________________________________________________________________ test_single_pytest __________________________________________________________________________________________________________________

    def test_single_pytest():
>       assert inc_dec.increment(-1) == 2
E       assert 0 == 2
E        +  where 0 = <function increment at 0x101512dc0>(-1)
E        +    where <function increment at 0x101512dc0> = inc_dec.increment

test_unit_and_pytest_combo.py:6: AssertionError
_____________________________________________________________________________________________________ test_class_unittest.test2_function_unittest _____________________________________________________________________________________________________

self = <test_unittest.test_class_unittest testMethod=test2_function_unittest>

    def test2_function_unittest(self):
>       self.assertEqual(inc_dec.decrement(7), 3, "Should be 6")
E       AssertionError: 6 != 3 : Should be 6

outer_nested_folder/inner_nested_folder/test_unittest.py:10: AssertionError
=============================================================================================================== short test summary info ===============================================================================================================
FAILED test_many_functions.py::test_many_2f - assert 4 == -1
FAILED test_many_functions.py::test_many_3f - assert 0 == 2
FAILED test_single_function.py::test_single_1f - assert 0 == 2
FAILED test_subtests.py::NumbersTest::test_even - AssertionError: 1 != 0
FAILED test_two_classes.py::ClassA::test_a2 - AssertionError: 6 != 3 : Should be 6
FAILED test_two_classes.py::ClassB::test_b2 - AssertionError: 6 != 3 : Should be 6
FAILED test_unit_and_pytest_combo.py::test_single_pytest - assert 0 == 2
FAILED outer_nested_folder/inner_nested_folder/test_unittest.py::test_class_unittest::test2_function_unittest - AssertionError: 6 != 3 : Should be 6
============================================================================================================ 8 failed, 27 passed in 3.87s =============================================================================================================

[eleanorjboyd]

postponing looking at this until next week since I have a PR related to changing how output looks which is pretty substantially. Will circle back once that is done on this, thanks

[eleanorjboyd]

fixes put in on insiders