Ior assignment operator is flagged while equivalent __ior__() method call is not

[kenmanheimer]

Bug Report

A |= operation statement causes "Result type of | incompatible in assignment" while the equivalent method call is accepted without complaint. I haven't figured out to annotate the statement to prevent that complaint, and expect that the complaint is an error.

To Reproduce

I am composing an OrderedSet based on one of Ramond Hettinger's recipes (the weakref one), adding needed functionality and type annotations. As in the recipe, my OrderedSet class is based on collections.MutableSet. Adding an update method, this definition results in Result type of | incompatible in assignment on the "|=" statement:

def update(self, *others):
        # type: (*AbstractSet) -> None
        for o in others:
            self |= o   # Mypy Error: Result type of | incompatible in assignment

However, code for the method that is identical except for using the corresponding method call instead of the |= operator does not cause any complaints:

def update(self, *others):
        # type: (*AbstractSet) -> None
        for o in others:
            self.__ior__(o)   # No error.

I would prefer to use the |= statement, but have been unable to annotate the unaccepted statement in a way that prevents the complaint (short of using ignore). (There are other methods that suffer from similar obstacles, including the original __init__() included in Raymond's recipe. The example I'm using involves fewer incidental details.)

Expected Behavior

I believe the |= statement should be accepted without complaint.

Actual Behavior

Result type of | incompatible in assignment error on the "|=" statement.

Your Environment

• Mypy version used: mypy 0.902
• Python version used: Python 3.7.5
• Operating system and version: Ubuntu Linux 18.04

[erictraut]

The statement self |= o should be translated to self = self.__ior__(o) in your second example. If you add the missing assignment, does the same error appear in both samples?

How is the __ior__ method annotated? If it's feasible for you to include a complete, self-contained sample, that would be helpful.

[kenmanheimer]

The statement self |= o should be translated to self = self.__ior__(o) in your second example. If you add the missing assignment, does the same error appear in both samples?

The self.__ior__(o) does achieve the intended result, as is. However, it still works if I do add the assignment you're suggesting, and it then elicits a different mypy error: Incompatible types in assignment (expression has type "MutableSet[Any]", variable has type "OrderedSet"). I can mitigate that by casting to OrderedSet: self = cast(OrderedSet, self.__ior__(o)). Trying something similar doesn't help with the |= ior assignment.

How is the __ior__ method annotated? If it's feasible for you to include a complete, self-contained sample, that would be helpful.

I'm pasting a current version of the module. I'm still shaking it out, but it largely works and has no few annotation errors. (I use comment-based annotation because I'm maintaining Python 2 compatibility, though Python 3 is my primary platform and none of these issues are particularly related to Python 2.)

"""A set that preserves insertion order.

Based on ordered set using weakrefs by Raymond Hettinger
Version: 2009-03-19
Licence: http://opensource.org/licenses/MIT
Source: http://code.activestate.com/recipes/576696/

Docstrings based on Emil Wall's contribution to that recipe.

Type annotations added by ken.manheimer@gmail.com.
"""

import collections
from weakref import proxy, WeakSet

from .sph_types_utils import Optional, Iterable, Iterator, Hashable, Dict, \
    cast, AbstractSet


class Link(object):
    root = None                 # type: Link
    prev = None                 # type: Link
    next = None                 # type: Link
    key = None                  # type: Link
#    __slots__ = 'prev', 'next', 'key', '__weakref__'


class OrderedSet(collections.MutableSet):
    """A set that preserves insertion order."""
    # Big-O running times for all methods are the same as for regular sets.
    # The internal self._map dictionary maps keys to links in a doubly linked
    # list.
    # The circular doubly linked list starts and ends with a sentinel element.
    # The sentinel element never gets deleted (this simplifies the algorithm).
    # The prev/next links are weakref proxies (to prevent circular references).
    # Individual links are kept alive by the hard reference in self._map.
    # Those hard references disappear when a key is deleted from an OrderedSet.

    def __init__(self, iterable=None):
        # type: (Optional[Iterable[Hashable]]) -> None
        """Create an ordered set.

        `iterable` -- A mutable iterable, typically a list or a set, containing
        initial values of the set. By default the initialized set is empty.
        """
        self._root = root = Link()  # sentinel node for doubly linked list
        root.prev = root.next = root
        self._map = {}              # type: Dict[Hashable, Link]
        if iterable is not None:
            # self |= iterable        # can't adquately annotate for mypy. )-:
            self.__ior__(cast(AbstractSet, iterable))

    def __len__(self):
        # type: () -> int
        """The number of items."""
        return len(self._map)

    def __contains__(self, key):
        # type: (Hashable) -> bool
        """True iff item is in this set."""
        return key in self._map

    def add(self, key):
        # type: (Hashable) -> None
        """Add `key` to the end of this set if not already present."""
        # Store new key in a new link at the end of the linked list
        if key not in self._map:
            self._map[key] = link = Link()
            root = self._root
            last = root.prev
            link.prev, link.next, link.key = last, root, key
            last.next = root.prev = proxy(link)

    def copy(self):
        # type: () -> OrderedSet
        """Return a copy of self with the same elements in the same order.
        """
        return self.__class__(self)

    def discard(self, key):
        # type: (Hashable) -> None
        """Discard `key` from this set."""
        # Remove an existing item using self._map to find the link which is
        # then removed by updating the links in the predecessor and successors.
        # Amortized O(1) time.
        if key in self._map:
            link = self._map.pop(key)
            link.prev.next = link.next
            link.next.prev = link.prev

    def difference(self, *others):
        # type: (*AbstractSet) -> OrderedSet
        """Return a new OrderedSet with our elements that are not in `others`.

        Order is preserved.
        """
        got = OrderedSet(self)
        for o in others:
            got -= o
        return got

    def union(self, *others):
        # type: (*AbstractSet) -> OrderedSet
        """Return a new OrderedSet having the elements of self and `others`.

        Order is preserved.
        """
        got = OrderedSet(self)
        for o in others:
            # got &= o            # can't adquately annotate for mypy. )-:
            got.__iand__(o)
        return got

    def update(self, *others):
        # type: (*AbstractSet) -> None
        """Add elements of `others` not present in self.

        Order is preserved.
        """
        for o in others:
            # self |= o            # can't adquately annotate for mypy. )-:
            self = cast(OrderedSet, self.__ior__(o))

    def __iter__(self):
        # type: () -> Iterator[Hashable]
        """Iterate over our contents in order.
        """
        # Traverse the linked list in order.
        root = self._root
        curr = root.next
        while curr is not root:
            yield curr.key
            curr = curr.next

    def __reversed__(self):
        # type: () -> Iterator[Hashable]
        """Iterate over our contents in reverse order.
        """
        root = self._root
        curr = root.prev
        while curr is not root:
            yield curr.key
            curr = curr.prev

    def pop(self, last=True):
        # type: (bool) -> Hashable
        """Discard the first or last element, returing it."""
        if not self:
            raise KeyError('set is empty')
        key = next(reversed(self)) if last else next(iter(self))
        self.discard(key)
        return cast(Hashable, key)

    def __repr__(self):
        # type: () -> str
        if not self:
            return '%s()' % (self.__class__.__name__,)
        return '%s(%r)' % (self.__class__.__name__, list(self))

    def __eq__(self, other):
        """True if `other` has the same elements and, if `other` is an
        OrderedSet, if the elements are in the same same order.
        """
        if isinstance(other, OrderedSet):
            return len(self) == len(other) and list(self) == list(other)
        else:
            return not self.isdisjoint(other)


class OrderedWeakrefSet(WeakSet):
    """Taken from Raymond Hettinger's solution:
    https://stackoverflow.com/a/7829569.
    """
    def __init__(self, values=()):
        super(OrderedWeakrefSet, self).__init__()
        self.data = OrderedSet()
        for elem in values:
            self.add(elem)

[kenmanheimer]

I noticed a mistake in my last post. There is an annotation error in the difference() method, of the same sort this issue is about, in this case: Result type of - incompatible in assignment

In any case, thanks for investigating this.

def difference(self, *others):
    # type: (*AbstractSet) -> OrderedSet
    """Return a new OrderedSet with our elements that are not in `others`.

    Order is preserved.
    """
    got = OrderedSet(self)
    for o in others:
        got -= o
    return got