Add more powerful set assertion rewrites for comparisons other than equality == (eg: <=, etc.)

[dougthor42]

What's the problem this feature will solve?

Set comparisons that do not check for pure equality do not have nice assertion rewriting.

Examples:

# Set equality assertion message is nice! 👍
>       assert {1, 2, 3} == {1, 2, 4}
E       assert {1, 2, 3} == {1, 2, 4}
E         Extra items in the left set:
E         3
E         Extra items in the right set:
E         4
E         Use -v to get more diff

# But others are not nice... 👎
# inequality
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}

# proper subset
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}

# proper superset
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}

# superset
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}

# subset
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}

Describe the solution you'd like

When asserting sets are unequal, such as set_a <= set_b, the assertion text should be more intelligent only displaying the values that fail the check.

This might look like:

# inequality
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}
E         Sets are equal

# proper subset
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}
E         Sets are equal

# proper superset
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}
E         Sets are equal

# superset
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}
E         Extra items in the right set:
E         4
E         Use -v to get more diff

# subset
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}
E         Extra items in the left set:
E         3
E         Use -v to get more diff

Alternative Solutions

I'm not aware of any plugins or other tools that implement this, but I have creating my own using pytest_assertrepr_compare:

def pytest_assertrepr_compare(op, left, right):
    if isinstance(left, set) and isinstance(right, set) and op == "!=":
        return [
            f"{left} {op} {right}",  # N.B.: I know this isn't the _actual_ way to do this, but it's good enough for the example!
            "Sets are equal"
        ]
    if isinstance(left, set) and isinstance(right, set) and op == "<=":
        extra = map(str, left - right)
        return [
            f"{left} {op} {right}",
            "Extra items in left set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == "<":
        if left == right:
            return [
                f"{left} {op} {right}",
                "Sets are equal"
            ]
        extra = map(str, left - right)
        return [
            f"{left} {op} {right}",
            "Extra items in left set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == ">=":
        extra = map(str, right - left)
        return [
            f"{left} {op} {right}",
            "Extra items in right set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == ">":
        if left == right:
            return [
                f"{left} {op} {right}",
                " Sets are equal",
            ]
        extra = map(str, right - left)
        return [
            f"{left} {op} {right}",
            "Extra items in right set:",
            *extra,
        ]

We then get the following:

$ pytest pytest_set_assertion.py 
======================================== test session starts =========================================
platform linux -- Python 3.9.13, pytest-7.2.0, pluggy-1.0.0
rootdir: /usr/local/google/home/dthor/dev/pytest_set_assertrepr
collected 8 items                                                                                    

pytest_set_assertion.py FFFFFFFF                                                               [100%]

============================================== FAILURES ==============================================
___________________________________________ test_equality ____________________________________________

    def test_equality():
>       assert {1, 2, 3} == {1, 2, 4, 5}
E       assert {1, 2, 3} == {1, 2, 4, 5}
E         Extra items in the left set:
E         3
E         Extra items in the right set:
E         4
E         5
E         Use -v to get more diff

pytest_set_assertion.py:2: AssertionError
__________________________________________ test_inequality ___________________________________________

    def test_inequality():
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}
E         Sets are equal

pytest_set_assertion.py:6: AssertionError
____________________________________________ test_subset _____________________________________________

    def test_subset():
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}
E         Extra items in left set:
E         3

pytest_set_assertion.py:10: AssertionError
_________________________________________ test_proper_subset _________________________________________

    def test_proper_subset():
>       assert {1, 2, 3} < {1, 2, 4}
E       assert {1, 2, 3} < {1, 2, 4}
E         Extra items in left set:
E         3

pytest_set_assertion.py:14: AssertionError
___________________________________ test_proper_subset_equal_sets ____________________________________

    def test_proper_subset_equal_sets():
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}
E         Sets are equal

pytest_set_assertion.py:18: AssertionError
___________________________________________ test_superset ____________________________________________

    def test_superset():
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}
E         Extra items in right set:
E         4

pytest_set_assertion.py:22: AssertionError
________________________________________ test_proper_superset ________________________________________

    def test_proper_superset():
>       assert {1, 2, 3} > {1, 2, 4}
E       assert {1, 2, 3} > {1, 2, 4}
E         Extra items in right set:
E         4

pytest_set_assertion.py:26: AssertionError
__________________________________ test_proper_superset_equal_sets ___________________________________

    def test_proper_superset_equal_sets():
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}
E          Sets are equal

pytest_set_assertion.py:30: AssertionError
====================================== short test summary info =======================================
FAILED pytest_set_assertion.py::test_equality - assert {1, 2, 3} == {1, 2, 4, 5}
FAILED pytest_set_assertion.py::test_inequality - assert {1, 2, 3} != {1, 2, 3}
FAILED pytest_set_assertion.py::test_subset - assert {1, 2, 3} <= {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_subset - assert {1, 2, 3} < {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_subset_equal_sets - assert {1, 2, 3} < {1, 2, 3}
FAILED pytest_set_assertion.py::test_superset - assert {1, 2, 3} >= {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_superset - assert {1, 2, 3} > {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_superset_equal_sets - assert {1, 2, 3} > {1, 2, 3}
========================================= 8 failed in 0.06s ==========================================

Additional context

• Python 3.9.13 on Debian
• pytest 7.2.0