c
single: Modernization
Before porting a C extension to Python 3, you'll need to make sure that you're not using features deprecated even in Python 2. Also, many of Python 3's improvements have been backported to Python 2.6, and using them will make the porting process easier.
For all changes you do, be sure add tests to ensure you do not break anything.
double: Modernization; Comparisons
Python 2.1 introduced rich comparisons for custom objects, allowing separate behavior for the ==
, !=
, <
, >
, <=
, >=
operators, rather than calling one __cmp__
function and interpreting its result according to the requested operation. (See PEP 207 for details.)
In Python 3, the original __cmp__
-based object comparison is removed, so all code needs to switch to rich comparisons. Instead of a :
static int cmp(PyObject *obj1, PyObject *obj2)
function in the tp_compare
slot, there is now a :
static PyObject* richcmp(PyObject *obj1, PyObject *obj2, int op)
in the tp_richcompare
slot. The op
argument specifies the comparison operation: Py_EQ (==), Py_GT (>), Py_LE (<=), etc.
Additionally, Python 3 brings a semantic change. Previously, objects of disparate types were ordered according to type, where the ordering of types was undefined (but consistent across, at least, a single invocation of Python). In Python 3, objects of different types are unorderable. It is usually possible to write a comparison function that works for both versions by returning NotImplemented to explicitly fall back to default behavior.
To help porting from __cmp__
operations, py3c defines a convenience macro, PY3C_RICHCMP, which evaluates to the right PyObject * result based on two values orderable by C's comparison operators. A typical rich comparison function will look something like this:
static PyObject* mytype_richcmp(PyObject *obj1, PyObject *obj2, int op)
{
if (mytype_Check(obj2)) {
return PY3C_RICHCMP(get_data(obj1), get_data(obj2), op);
}
Py_RETURN_NOTIMPLEMENTED;
}
where get_data
returns an orderable C value (e.g. a pointer or int), and mytype_Check checks if get_data
is of the correct type (usually via PyObject_TypeCheck). Note that the first argument, obj1, is guaranteed to be of the type the function is defined for.
If a "cmp"-style function is provided by the C library, use PY3C_RICHCMP(mytype_cmp(obj1, obj2), 0, op)
.
Also, py3c defines the Py_RETURN_NOTIMPLEMENTED macro if it's not provided by your Python version (3.3 and lower).
Note that if you use PY3C_RICHCMP, you will need to include the header py3c/comparison.h
(or copy the macro to your code) even after your port to Python 3 is complete. The is also needed for Py_RETURN_NOTIMPLEMENTED until you drop support for Python 3.3.
Note
The tp_richcompare
slot is inherited in subclasses together with tp_hash
and (in Python 2) tp_compare
: iff the subclass doesn't define any of them, all are inherited.
This means that if a class is modernized, its subclasses don't have to be, unless the subclass manipulates compare/hash slots after class creation (e.g. after the :cPyType_Ready <py3:PyType_Ready>
call).
double: Modernization; PyObject structure double: Modernization; Objects
To conform to C's strict aliasing rules, PyObject_HEAD, which provides members such as ob_type
and ob_refcnt
, is a separate struct in Python 3. Access to these members is provided by macros, which have been ported to Python 2.6:
Instead of | use |
---|---|
obj->ob_type | Py_TYPE(obj) |
obj->ob_refcnt | Py_REFCNT(obj) |
obj->ob_size | Py_SIZE(obj) |
And for initialization of type objects, the sequence :
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
must be replaced with :
PyVarObject_HEAD_INIT(NULL, 0)
double: Modernization; Constants
Often, module initialization uses code like this:
PyModule_AddObject(m, "RDWR", PyInt_FromLong(O_RDWR));
PyModule_AddObject(m, "__version__", PyString_FromString("6.28"));
Python 2.6 introduced convenience functions, which are shorter to write:
PyModule_AddIntConstant(m, "RDWR", O_RDWR)
PyModule_AddStringConstant(m, "__version__", "6.28")
These will use native int and str types in both Python versions.
double: Modernization; Classes
The old-style classes (PyClass_*
and PyInstance_*
) will be removed in Python 3. Instead, use type objects, which have been available since Python 2.2.
double: Modernization; PyCObject double: Modernization; PyCapsule
The PyCObject API has been removed in Python 3.3. You should instead use its replacement, PyCapsule, which is available in Python 2.7 and 3.1+. For the rationale behind Capsules, see CPython issue 5630.
If you need to support Python 2.6, you can use capsulethunk.h
, which implements the PyCapsule API (with some limitations) in terms of PyCObject. For instructions, see the chapter capsulethunk
.
The port to PyCapsule API should be straightforward:
- Instead of :c
PyCObject_FromVoidPtr(obj, destr) <py2:PyCObject_FromVoidPtr>
, use :cPyCapsule_New(obj, name, destr) <py3:PyCapsule_New>
. If the capsule will be available as a module attribute, use"<modulename>.<attrname>"
for name. Otherwise, use your best judgment, but try making the name unique. - Instead of :c
PyCObject_FromVoidPtrAndDesc(obj, desc, destr) <py2:PyCObject_FromVoidPtrAndDesc>
, use :cpy3:PyCapsule_New
as above; then call :cPyCapsule_SetContext(obj, desc) <py3:PyCapsule_SetContext>
. - Instead of :c
PyCObject_AsVoidPtr(obj) <py2:PyCObject_AsVoidPtr>
, use :cPyCapsule_GetPointer(obj, name) <py3:PyCapsule_GetPointer>
. You will need to provide a capsule name, which is checked at runtime as a form of type safety. - Instead of :c
py2:PyCObject_GetDesc
, use :cpy3:PyCapsule_GetContext
. - Instead of :c
py2:PyCObject_SetVoidPtr
, use :cpy3:PyCapsule_SetPointer
. - Change all CObject destructors to :c
PyCapsule destructors <py3:PyCapsule_Destructor>
, which take the PyCapsule object as their only argument.
When your project is sufficiently modernized, and the tests still pass under Python 2, you're ready to start the actual Porting <guide-porting>
.