Caching error (Metaclass conflict) with mypy 0.991

[euresti]

Hi. With the latest version of mypy (0.991) and mypy-zope (0.3.11) we're getting mypy errors that I think are cache related. The error goes away after rm -rf .mypy_cache

Here's the easiest repro I could make:

iface.py

from zope.interface import Interface
from zope.interface import implementer

class ISomething(Interface):
    pass

class ISomething2(Interface):
    pass

@implementer(ISomething)
class OneInterface:
    pass

@implementer(ISomething, ISomething2)
class TwoInterfaces:
    pass

foo.py

from iface import OneInterface, TwoInterfaces

class Good(OneInterface):
    pass

class Bad(TwoInterfaces):
    pass

I am able to reliably hit the problem by running the following commands:

$ rm -rf .mypy_cache
$ mypy iface.py 
Success: no issues found in 1 source file
$ mypy foo.py 
foo.py:6: error: Metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases  [misc]
Found 1 error in 1 file (checked 1 source file)

As you can tell this only happens when there are multiple interfaces.

[glyph]

I also encountered an apparently cache-related issue, and got this internal traceback:

/Users/glyph/.virtualenvs/Pomodouroboros/lib/python3.10/site-packages/twisted/_threads/_threadworker.py:19: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.991
Traceback (most recent call last):
  File "mypy/semanal.py", line 6047, in accept
  File "mypy/nodes.py", line 1143, in accept
  File "mypy/semanal.py", line 1435, in visit_class_def
  File "mypy/semanal.py", line 1511, in analyze_class
  File "mypy/semanal.py", line 1538, in analyze_class_body_common
  File "mypy/semanal.py", line 1602, in apply_class_plugin_hooks
  File "/Users/glyph/.virtualenvs/Pomodouroboros/lib/python3.10/site-packages/mypy_zope/plugin.py", line 301, in analyze
    apply_implementer(iface_arg, classdef_ctx.cls.info, api)
  File "/Users/glyph/.virtualenvs/Pomodouroboros/lib/python3.10/site-packages/mypy_zope/plugin.py", line 293, in apply_implementer
    self._apply_interface(class_info, iface_type)
  File "/Users/glyph/.virtualenvs/Pomodouroboros/lib/python3.10/site-packages/mypy_zope/plugin.py", line 700, in _apply_interface
    faketi._promote = promote
TypeError: list object expected; got mypy.types.Instance
/Users/glyph/.virtualenvs/Pomodouroboros/lib/python3.10/site-packages/twisted/_threads/_threadworker.py:19: : note: use --pdb to drop into pdb

This revision of this repo, if it helps:
https://github.com/glyph/Pomodouroboros/tree/28da7911163e7dc9d52f75951c9bb3b13c1f42b5

[kedder]

I think 0.991 is not yet supported by mypy-zope. There are some problems (in #82) that we haven't found a workaround for yet.

[euresti]

Ok I did a little research into this and I see 2 bugs. But fixing both wouldn't actually fix the issue. :)

The thing that is emitting the error is check_metaclass_compatibility

In particular the line:

        metaclasses = [
            entry.metaclass_type
            for entry in typ.mro[1:-1]
            if entry.metaclass_type
            and not is_named_instance(entry.metaclass_type, "builtins.type")
        ]

returns [zope.interface.interface.InterfaceClass]
but typ.metaclass_type is None so we fail.

Now bug #1 is that it only fails for two or more Interfaces. This is because we append the types to the MRO like this:
impl.mro.append(faketi). But the code above omits the last one, assuming that it's a builtins.object. This can be fixed by changing the line to impl.mro.insert(len(impl.mro) - 1, faketi). Great we now made it fail for all cases. :)

Bug #2 is that we append a faketi to the mro in the first pass. This faketi doesn't have a metaclass_type set and so check_metaclass_compatibility passes.

However in the cache mypy ends up writing references like this:

"mro": [
          "iface.TwoInterfaces",
          "builtins.object",
          "iface.ISomething",
          "iface.ISomething2"
        ],

And when it loads from the cache the real ISomething type info is loaded which does have the metaclass_type set.
We could ensure that the metaclass_type is set on the faketi but that would cause the error to happen every time instead of just the 2nd time.

So I'm not sure where to go from here. The only thing I can think of is somehow setting the metaclass_type of Good and Bad to zope.interface.interface.InterfaceClass but that seems incorrect.

[euresti]

I do think that this is a bigger problem than #82 which a) Doesn't affect everyone (since it only affects projects that create Interfaces not that just use them) and b) I believe you can temporarily add a # type: ignore on the specific line until this issue gets solved.