Attempt to make the test suite pass under mypy 1.0.0 #89

DMRobertson · 2023-02-08T18:30:19Z

For your consideration: an attempt to update this plugin to work with the recent mypy 1.0.0 release.

#85 was an attempt to establish compatibility with mypy 0.991. It was partially successful, but ultimately blocked on python/mypy#14106

I wanted to take a look to see if could make any progress. I think I was able to workaround the linked mypy issue with a dirty, brittle and naughty hack (see da1d0fb). It might be too brittle to include upstream; let me know what you think.

On my machine, all but one test passes. I am not sure how to handle the failure:

$ pytest -vv --lf
=================================================================================================================================== test session starts ====================================================================================================================================
platform linux -- Python 3.10.9, pytest-7.1.3, pluggy-1.0.0 -- /home/dmr/workspace/mypy-zope/ve/bin/python3
cachedir: .pytest_cache
rootdir: /home/dmr/workspace/mypy-zope
plugins: cov-3.0.0
collected 1 item                                                                                                                                                                                                                                                                           
run-last-failure: rerun previous 1 failure (skipped 1 file)

tests/test_samples.py::test_samples[interface_meta] FAILED                                                                                                                                                                                                                           [100%]

========================================================================================================================================= FAILURES =========================================================================================================================================
_______________________________________________________________________________________________________________________________ test_samples[interface_meta] _______________________________________________________________________________________________________________________________

samplefile = '/home/dmr/workspace/mypy-zope/tests/samples/interface_meta.py', mypy_cache_dir = '/tmp/pytest-of-dmr/pytest-75/.mypy_cahe0'

    def test_samples(samplefile, mypy_cache_dir):
        opts = options.Options()
        opts.cache_dir = mypy_cache_dir
        opts.show_traceback = True
        opts.namespace_packages = True
        opts.hide_error_codes = True
        opts.plugins = ['mypy_zope:plugin']
        # Config file is needed to load plugins, it doesn't not exist and is not
        # supposed to.
        opts.config_file = '    not_existing_config.ini'
    
        try:
            base_dir = os.path.dirname(samplefile)
            source = BuildSource(samplefile,
                                 module=None,
                                 text=None,
                                 base_dir=base_dir)
            res = build.build(
                sources=[source],
                options=opts)
        except CompileError as e:
            assert False, e
    
        normalized = normalize_errors(res.errors, samplefile)
        actual = '\n'.join(normalized)
        expected = find_expected_output(samplefile)
>       assert actual == expected
E       assert ('interface_meta.py:13: error: Missing positional argument "object" in call to '\n '"providedBy" of "ISpecification"\n'\n 'interface_meta.py:13: error: Argument 1 to "providedBy" of "ISpecification" '\n 'has incompatible type "Type[T]"; expected "ISpecification"\n'\n 'interface_meta.py:19: note: Revealed type is "__main__.IBookmark"') == 'interface_meta.py:19: note: Revealed type is "__main__.IBookmark"'
E         + interface_meta.py:13: error: Missing positional argument "object" in call to "providedBy" of "ISpecification"
E         + interface_meta.py:13: error: Argument 1 to "providedBy" of "ISpecification" has incompatible type "Type[T]"; expected "ISpecification"
E           interface_meta.py:19: note: Revealed type is "__main__.IBookmark"

tests/test_samples.py:49: AssertionError
---------------------------------------------------------------------------------------------------------------------------------- Captured stdout setup -----------------------------------------------------------------------------------------------------------------------------------
Setup cache /tmp/pytest-of-dmr/pytest-75/.mypy_cahe0
================================================================================================================================= short test summary info ==================================================================================================================================
FAILED tests/test_samples.py::test_samples[interface_meta] - assert ('interface_meta.py:13: error: Missing positional argument "object" in call to '\n '"providedBy" of "ISpecification"\n'\n 'interface_meta.py:13: error: Argument 1 to "providedBy" of "ISpecification" '\n 'has incom...
==================================================================================================================================== 1 failed in 3.92s =====================================================================================================================================

Fixes #82 (I think?)
Closes #85

To avoid having to update all the samples

To stop mypy from complaining that they have empty bodies

Implicit optionals are no longer accepted by mypy 1.0

Since python/mypy#13729, mypy complains if a non-abstract function has a trivial body. Avoid this providing a dummy body.

kedder · 2023-02-08T19:39:50Z

src/mypy_zope/plugin.py

+        # Thus we can make (1) and (2) true by setting abstract_status to some value
+        # distinct from these three NOT_ABSTRACT, IS_ABSTRACT and IMPLICITLY_ABSTRACT.
+        # These are presently the integers 0, 1, and 2 defined in mypy/nodes.py:738-743.
+        func_def.abstract_status = HACK_IS_ABSTRACT_NON_PROPAGATING


This is very unorthodox, but if it will let us to upgrade to mypy-1.0, I'm all for it! :)

kedder · 2023-02-08T19:59:03Z

src/mypy_zope/plugin.py

@@ -479,6 +482,9 @@ def _analyze_zope_interface(
                replacement = self._adjust_interface_function(api, cls.info, item)
            elif isinstance(item, OverloadedFuncDef):
                replacement = self._adjust_interface_overload(api, cls.info, item)
+            elif isinstance(item, Decorator):
+                replacement = item
+                replacement.func = self._adjust_interface_function(api, cls.info, item.func)


This seems to be causing error in the description. What this change is designed for?

The following patch to your branch makes all tests pass:

diff --git a/src/mypy_zope/plugin.py b/src/mypy_zope/plugin.py index e9aeca7..02ab413 100644 --- a/src/mypy_zope/plugin.py +++ b/src/mypy_zope/plugin.py @@ -482,9 +482,6 @@ class ZopeInterfacePlugin(Plugin): replacement = self._adjust_interface_function(api, cls.info, item) elif isinstance(item, OverloadedFuncDef): replacement = self._adjust_interface_overload(api, cls.info, item) - elif isinstance(item, Decorator): - replacement = item - replacement.func = self._adjust_interface_function(api, cls.info, item.func) else: continue diff --git a/tests/samples/interface_meta.py b/tests/samples/interface_meta.py index 2e04617..94d63fa 100644 --- a/tests/samples/interface_meta.py +++ b/tests/samples/interface_meta.py @@ -7,7 +7,7 @@ T = TypeVar('T', bound='zope.interface.Interface') class IBookmark(zope.interface.Interface): @staticmethod def create(url: str) -> 'IBookmark': - pass + return IBookmark(url) def createOb(iface: Type[T]) -> T: if zope.interface.interfaces.IInterface.providedBy(iface):

Well, actually my patch is stupid, interface cannot have an implementation, disregard it...

That create method didn't make too much sense anyways. I suggest this, unless there's something important about handling decorators:

diff --git a/src/mypy_zope/plugin.py b/src/mypy_zope/plugin.py index e9aeca7..02ab413 100644 --- a/src/mypy_zope/plugin.py +++ b/src/mypy_zope/plugin.py @@ -482,9 +482,6 @@ class ZopeInterfacePlugin(Plugin): replacement = self._adjust_interface_function(api, cls.info, item) elif isinstance(item, OverloadedFuncDef): replacement = self._adjust_interface_overload(api, cls.info, item) - elif isinstance(item, Decorator): - replacement = item - replacement.func = self._adjust_interface_function(api, cls.info, item.func) else: continue diff --git a/tests/samples/interface_meta.py b/tests/samples/interface_meta.py index 2e04617..35a7d42 100644 --- a/tests/samples/interface_meta.py +++ b/tests/samples/interface_meta.py @@ -5,8 +5,6 @@ T = TypeVar('T', bound='zope.interface.Interface') class IBookmark(zope.interface.Interface): - @staticmethod - def create(url: str) -> 'IBookmark': pass def createOb(iface: Type[T]) -> T: @@ -20,6 +18,6 @@ def main(self) -> None: """ <output> -interface_meta.py:19: note: Revealed type is "__main__.IBookmark" +interface_meta.py:17: note: Revealed type is "__main__.IBookmark" </output> """

What this change is designed for?

The IBookmark example that you cite. I thought the staticmethod was intended to be part of the IBookmark interface, so that classes which implement it would have to define their own create method.

If that's not the case then let's apply your patch!

I don't even think interfaces can define static methods... At least it doesn't make sense to me today.

I suggest this, unless there's something important about handling decorators:

Ahh I see, the test doesn't even make use of create. Let's try the second patch.

kedder

This looks good to me. Take it out of WIP whenever you're ready to merge this.

And thanks for digging into it!

DMRobertson · 2023-02-09T16:15:10Z

Take it out of WIP whenever you're ready to merge this.

I've just done so. (I wanted to run this PR + mypy 1.0 against https://github.com/matrix-org/synapse/. There weren't any new errors related to the plugin AFACIS, other than those covered by #88.)

And thanks for digging into it!

My pleasure. I just hope the hack doesn't come back to bite us in the future!

kedder · 2023-02-09T20:42:33Z

I just hope the hack doesn't come back to bite us in the future!

I'm sure it will eventually, but we'll deal with it when it happens :)

This fixes the regression when #88 combined with #89: mypy-1.0 started producing the error: ``` Metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases ```

kedder · 2023-02-10T06:26:37Z

Released in mypy-zope-0.9.0

David Robertson added 7 commits February 8, 2023 18:19

Test against mypy 1.0.0

fb1accd

Hide error codes

3877ca9

To avoid having to update all the samples

Mark interface method definitions as abstract

da1d0fb

To stop mypy from complaining that they have empty bodies

Update interface_unknown_inherit error message

2bd8dfd

Fix overload-readme sample

7c42399

Implicit optionals are no longer accepted by mypy 1.0

Fix contextmanager sample

91ae304

Since python/mypy#13729, mypy complains if a non-abstract function has a trivial body. Avoid this providing a dummy body.

Mark decorated interface functions as abstract

d3697b8

DMRobertson changed the title ~~WIP: Attempt to make the test sutie pass under mypy 1.0.0~~ WIP: Attempt to make the test suite pass under mypy 1.0.0 Feb 8, 2023

kedder reviewed Feb 8, 2023

View reviewed changes

kedder added 3 commits February 8, 2023 20:12

Remove faulty decorator logic

4b4e929

Provide a nonempty body in interface_meta example

5e64237

Remove the staticmethod from IBookmark

7c07bff

DMRobertson marked this pull request as ready for review February 8, 2023 20:51

kedder approved these changes Feb 9, 2023

View reviewed changes

DMRobertson changed the title ~~WIP: Attempt to make the test suite pass under mypy 1.0.0~~ Attempt to make the test suite pass under mypy 1.0.0 Feb 9, 2023

kedder merged commit 88308ac into Shoobx:master Feb 9, 2023

kedder mentioned this pull request Feb 9, 2023

Fix the MRO order and correctly add metaclass types #88

Merged

kedder mentioned this pull request Feb 9, 2023

Fix metaclass conflict #90

Merged

DMRobertson mentioned this pull request Feb 10, 2023

Use mypy 1.0 matrix-org/synapse#15052

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Attempt to make the test suite pass under mypy 1.0.0 #89

Attempt to make the test suite pass under mypy 1.0.0 #89

DMRobertson commented Feb 8, 2023 •

edited by kedder

kedder Feb 8, 2023

kedder Feb 8, 2023

kedder Feb 8, 2023

kedder Feb 8, 2023

DMRobertson Feb 8, 2023

kedder Feb 8, 2023

DMRobertson Feb 8, 2023

kedder left a comment

DMRobertson commented Feb 9, 2023 •

edited

kedder commented Feb 9, 2023

kedder commented Feb 10, 2023

Attempt to make the test suite pass under mypy 1.0.0 #89

Attempt to make the test suite pass under mypy 1.0.0 #89

Conversation

DMRobertson commented Feb 8, 2023 • edited by kedder

kedder Feb 8, 2023

Choose a reason for hiding this comment

kedder Feb 8, 2023

Choose a reason for hiding this comment

kedder Feb 8, 2023

Choose a reason for hiding this comment

kedder Feb 8, 2023

Choose a reason for hiding this comment

DMRobertson Feb 8, 2023

Choose a reason for hiding this comment

kedder Feb 8, 2023

Choose a reason for hiding this comment

DMRobertson Feb 8, 2023

Choose a reason for hiding this comment

kedder left a comment

Choose a reason for hiding this comment

DMRobertson commented Feb 9, 2023 • edited

kedder commented Feb 9, 2023

kedder commented Feb 10, 2023

DMRobertson commented Feb 8, 2023 •

edited by kedder

DMRobertson commented Feb 9, 2023 •

edited