New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support metaclasses #2475

Merged
merged 18 commits into from Feb 7, 2017

Conversation

Projects
None yet
3 participants
@elazarg
Contributor

elazarg commented Nov 22, 2016

Reopening #2392 (itself a reopen of #2365).

Store the type of the metaclass in TypeInfo, and use it when possible, instead of builtins.type.

I assume metaclasses always inherit from type.

This PR will help fixing #2305 and #741, but the fix is not part of this PR.

@elazarg elazarg changed the title from Support metaclasses to [WIP] Support metaclasses Nov 22, 2016

@elazarg

This comment has been minimized.

Contributor

elazarg commented Nov 22, 2016

(No tests yet)

@JukkaL

This comment has been minimized.

Collaborator

JukkaL commented Nov 25, 2016

Looks reasonable -- thanks for working on this! One thing wasn't clear: does this follow the MRO in search of the metaclass?

Also, we may also want to check for incompatible metaclasses in the MRO, but that can be a separate issue and doesn't need to be covered here.

@JukkaL

This comment has been minimized.

Collaborator

JukkaL commented Nov 25, 2016

(I only did a quick pass, not a full review.)

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Dec 13, 2016

Hey @elazarg, are you still planning to work on this?

@elazarg elazarg force-pushed the elazarg:metaclasses branch from 59e1883 to 8ed204e Dec 13, 2016

[out]
main:2: error: Name 'abc.Foo' is not defined
main:2: error: Invalid metaclass 'abc.Foo'

This comment has been minimized.

@elazarg

elazarg Dec 13, 2016

Contributor

Which one is preferable?

This comment has been minimized.

@gvanrossum

gvanrossum Dec 13, 2016

Member

Given that abc.Foo does not exist, I prefer "Name 'abc.Foo' is not defined" -- if I were to see the other I would assume that there is an abc.Foo that was unacceptable as a metaclass. (Typos and large lists of imports might obscure that it's not defined at all.)

@elazarg

This comment has been minimized.

Contributor

elazarg commented Dec 13, 2016

Yes, sorry. Todo:

  • Add test
  • Find out why would self.mro be None
  • Follow mro better

Also, I should look for other places where the fallback is needed, other than for iterable. (I must note that the tests for fallbackness feels awkward and error prone)

@elazarg elazarg changed the title from [WIP] Support metaclasses to Support metaclasses Dec 14, 2016

@elazarg

This comment has been minimized.

Contributor

elazarg commented Dec 14, 2016

I think this is ready for a review.

def __iter__(self) -> Iterator[int]: yield 1

class Good(metaclass=GoodMeta): pass
for _ in Good: pass

This comment has been minimized.

@gvanrossum

gvanrossum Dec 14, 2016

Member

If this works, shouldn't list(C) work too?

This comment has been minimized.

@elazarg

elazarg Dec 14, 2016

Contributor

I just assumed it will work. Turns out it didn't, but after handling the fallback in the solver it does.

@elazarg elazarg force-pushed the elazarg:metaclasses branch from a91c0a1 to 7876d3f Feb 6, 2017


def calculate_metaclass_type(self) -> 'Optional[mypy.types.Instance]':
if self.mro is None:
# XXX why does this happen?

This comment has been minimized.

@gvanrossum

gvanrossum Feb 6, 2017

Member

Do you have a sample that triggers this condition?

Presumably the clue is the comment in __init__ on line 1940 above. Maybe it has something to do with the MRO being calculated twice -- the first time (in semanal.py on line 850) it may fail if there's an import cycle causing a base class being a forward reference; the second time on line 3128 in ThirdPass, called from build.py line 1440. (There's also a call from fixup.py, but I think that's not what you're seeing.)

This comment has been minimized.

@elazarg

elazarg Feb 6, 2017

Contributor

Turns out it happens during deserialization in incremental mode: the type of a direct base class may be None. (I should have looked into it long time ago. Sorry)

This comment has been minimized.

@elazarg

elazarg Feb 6, 2017

Contributor

I think I will remove the call from deserialize, since it shouldn't be done yet - it should be done from fixup.py. Does it sound reasonable?

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Feb 6, 2017

self.fail("Invalid metaclass '%s'" % defn.metaclass, defn)
return
inst = fill_typevars(sym.node)
assert isinstance(inst, Instance)

This comment has been minimized.

@gvanrossum

gvanrossum Feb 6, 2017

Member

I can trigger this assert as follows:

class M(Tuple[int]): pass
class C(metaclass=M): pass

This comment has been minimized.

@elazarg

elazarg Feb 6, 2017

Contributor

I think it should be an error then. Do you agree? I don't want to go silently to a fallback,

This comment has been minimized.

@gvanrossum

gvanrossum Feb 6, 2017

Member

Yes, you can add the check to line 911.

@gvanrossum

Could you also look into #1771? Possibly as a follow-up PR?

candidates = {s.declared_metaclass for s in self.mro} - {None}
for c in candidates:
if all(other.type in c.type.mro for other in candidates):
return c

This comment has been minimized.

@gvanrossum

gvanrossum Feb 6, 2017

Member

How sure are you that the answer does not depend on the order of candidates?

This comment has been minimized.

@elazarg

elazarg Feb 7, 2017

Contributor

Not at all. Perhaps redundant tests checks are better.

This comment has been minimized.

@elazarg

elazarg Feb 7, 2017

Contributor

I have replaced the set with a simple list

candidates = [s.declared_metaclass for s in self.mro if s.declared_metaclass is not None]
self.fail("Invalid metaclass '%s'" % defn.metaclass, defn)
return
inst = fill_typevars(sym.node)
assert isinstance(inst, Instance)

This comment has been minimized.

@gvanrossum

gvanrossum Feb 6, 2017

Member

Yes, you can add the check to line 911.


class A(metaclass=M): pass

reveal_type(A.x) # E: Revealed type is 'builtins.int'

This comment has been minimized.

@gvanrossum

gvanrossum Feb 6, 2017

Member

This line seems redundant, it's the same as in the previous test.

This comment has been minimized.

@elazarg

elazarg Feb 7, 2017

Contributor

I have removed it in the last commit but it still appears here.

@elazarg

This comment has been minimized.

Contributor

elazarg commented Feb 7, 2017

I will try to look into #1771 later. But it feels like it will require a hack (or a very deep change).

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Feb 7, 2017

@gvanrossum gvanrossum merged commit cab1e0d into python:master Feb 7, 2017

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Feb 7, 2017

Thanks. Great work!

JukkaL added a commit that referenced this pull request Feb 7, 2017

Calculate implicit metaclasses (#2819)
Fix an issue in #2475 - metaclasses are not calculated for subclasses (this 
is needed for Enum).

JukkaL added a commit that referenced this pull request Feb 7, 2017

Do not support metaclasses not inheriting from type (#2820)
Fix #2818.

ABCMeta requires special treatment.

Related: #2475

JukkaL added a commit that referenced this pull request Feb 7, 2017

Check for member in type-fallback (#2822)
Fix subissue of #2413.

Follow-up for #2475

* check for member in fallback

* check for fallback only for type objects
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment