Multiple Metaclass inheritance limitation

[glchapman]

BPO	963246
Nosy	@pjeby

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2004-06-05.20:19:49.000>
created_at = <Date 2004-05-30.19:52:23.000>
labels = ['interpreter-core']
title = 'Multiple Metaclass inheritance limitation'
updated_at = <Date 2004-06-05.20:19:49.000>
user = 'https://bugs.python.org/glchapman'

bugs.python.org fields:

activity = <Date 2004-06-05.20:19:49.000>
actor = 'pje'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2004-05-30.19:52:23.000>
creator = 'glchapman'
dependencies = []
files = []
hgrepos = []
issue_num = 963246
keywords = []
message_count = 2.0
messages = ['20929', '20930']
nosy_count = 2.0
nosy_names = ['glchapman', 'pje']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue963246'
versions = ['Python 2.3']

[glchapman]

I'm not sure if this constitutes a bug or a limitation
(which should be documented?), but the following
doesn't work with Python 2.3.4.

Assume CMeta is a type defined in C, with
tp_base = PyType_Type
tp_new = some C function which uses the mro to
call the inherited tp_new
tp_flags includes Py_TPFLAGS_BASETYPE

class PyMeta(type):
    def __new__(meta, name, bases, attrs):
        return super(PyMeta, meta).__new__(meta,
                       name, bases, attrs)

class MetaTest(CMeta, PyMeta):
    pass

class Test:
    __metaclass__ = MetaTest

The attempt to define Test generates a
TypeError: "type.__new__(MetaTest) is not safe, use
CMeta.__new__()".

The above error is generated (in tp_new_wrapper) by
the super call in PyMeta.__new__, but this is only
reached as a result of an initial call to CMeta.tp_new
(which, using the mro to find the next "__new__"
method, finds and calls PyMeta.__new__).

It may be there is no good way to allow the above
scenario, but I just thought I'd point it out in case
someone can think of a workaround.

[pjeby]

Logged In: YES
user_id=56214

There are two things that can cause the error message you
got. One is calling a Python __new__ from a C type, such as
your code is doing. The only workaround is "don't do that".
A C type must always call only C __new__ methods.

You can avoid this in your example by moving CMeta *after*
PyMeta in the __bases__, but it will fail if CMeta is ever
placed anywhere but the end of the list. The alternative is
to change CMeta to call its tp_base->tp_new instead of using
the mro to find the next base. This will silently ignore
any Python __new__ methods in the mro, instead of causing a
TypeError.

Another multiple inheritance situation that can cause the
same error is if you define a C type which subtypes another
C type and does not increase its tp_basicsize, *and* the C
type is placed anywhere but first in the __bases__ of a
Python subclass. You can work around that by ensuring that
its tp_basicsize is larger than that of its base C type, so
that Python will always pick it as the __base__ (aka
tp_base) even if it is not listed first in __bases__.

I have written the following additions to the Extending and
Embedding manual for future reference:

\note{If you want your type to be subclassable from Python,
and your
type has the same \member{tp_basicsize} as its base type,
you may
have problems with multiple inheritance. A Python subclass
of your
type will have to list your type first in its
\member{bases}, or
else it will not be able to call your type's
\method{new} method
without getting an error. You can avoid this problem by
ensuring
that your type has a larger value for \member{tp_basicsize} than
its base type does. Most of the time, this will be true anyway,
because either your base type will be \class{object}, or
else you will
be adding data members to your base type, and therefore
increasing its
size.}

and...

\note{If you are creating a co-operative \member{tp_new}
(one that
calls a base type's \member{tp_new} or \method{new}), you
must \emph{not} try to determine what method to call using
method resolution order at runtime. Always statically determine
what type you are going to call, and call its \member{tp_new}
directly, or via \code{type->tp_base->tp_new}. If you do
not do this, Python subclasses of your type that also inherit
from other Python-defined classes may not work correctly.
(Specifically, you may not be able to create instances of
such subclasses without getting a \exception{TypeError}.)}

For more discussion on this, you can also see:

A thread on Python-Dev that touches on these issues:
http://mail.python.org/pipermail/python-dev/2003-April/034633.html

Some notes on ZODB4 running afoul of the same issues:
http://collector.zope.org/Zope3-dev/86