Python3 port of the C-Extension #9

stephan-hof · 2016-09-30T16:07:07Z

As requested in #7 I open a pull request to port the C-Extension of ExtensionClass.

Basically do what python version does. Call object.__getattribute__ and if it's a descriptor for something besides __parent__, call it.

Py_TPFLAGS_HAVE_VERSION_TAG is available since python2.6

This macro works for python2 and python3.

Basically do the same as the python version. Get the super class and search for the attribute there.

pickle support in persistent is already python3 ready so reuse the implementation. The python version is also a copy of persistent and the previous C version look also like a copy&paste from persistent.

In python3 unbound method disappeared. So it is very likely that __class_init__ is just a function. => Allow functions also as __class_init__

convert_name takes care to convert unicode (python3) names into PyBytes. In python2 convert_name just checks if it's already a PyString. (PyBytes and PyString are the *same* in python2) So EC_setattro can rely on convert_name to emit always PyBytes.

…d py3.

The fact that unbound methods do not exist in python3 anymore make this a bit hard. Basically I cannot call 'callable->ob_type->tp_descr_get' anymore, because the callable has *no* im_class anymore. Yes I could have done it in the python2 case, but from my point of view it should behave consistent by not calling 'tp_descr_get' in in both versions.

Otherwise the import of python3 (which is absolute now), does not pick up the C-Extension.

This makes it easier to Py_DECREF(as_bytes) in case of errors.

tseaver · 2016-10-01T21:12:45Z

@stephan-hof Wow, thank you! I will try to get a review done this coming week (unless somebody else beats me to it).

tseaver · 2016-10-02T00:38:50Z

I haven't done a detailed review, but I did check that your branch shows 100% test coverage after merging with the current master (we're only measuring coverage of Python code, of course).

stephan-hof · 2016-10-04T08:02:23Z

I found a regression in ExtensionClass (python version and also my C version) while working on the C-Extension of Acquisition

The old C code of ExtensionClass changed an attribute error into the following:

class Dummy(Base):
    pass

Dummy().a
AttributeError: a

The python version (+ my ported C version) does the following:

class Dummy(Base):
    pass

Dummy().a
AttributeError: 'Dummy' object has no attribute 'a'

See, the error messages contains the typename of an object.
From my point of view a port should not change behaviour, so I would change the C and python version to hide the typename again.

The python version of Acquisition struggled also with the changed behaviour of ExtensionClass.
A lot of doctests there have asserts for AttributeError: <name> (no typename in the error message), but instead of questioning ExtensionClass a fix to Acquisition was introduced.
https://github.com/zopefoundation/Acquisition/blob/master/src/Acquisition/__init__.py#L745
When ExtensionClass behaves the old way Acquisition could be changed to rely on ExtensionClass to throw the appropriate AttributeError. This would make a port of the C-Extension of Acquisition easier too.

Alternative:

Keep the regression => Keep the new style Attribute error
Change all the doctests in Acquisition to assert the new style Attribute error
Change python version of Acquisition to keep the new style Attribute error

@jamadden any comments on this? Since you wrote the python version of Acquisition.

jamadden · 2016-10-04T10:53:20Z

I don't particularly care. The only reason the code in Acquisition you pointed to is there is because of doctests; I'm not particularly a fan of doctests or relying on exact exception messages in general, but I didn't want to break anything or create too large of a PR.

Personally, I'd rather see the more informative error messages raised everywhere (although in Acquisition's case, that could be slightly misleading).

tseaver · 2016-10-04T16:28:27Z

We have no guarantee that Python itself doesn't break reliance on exact exception message tests: feel free to break doctests which rely on them if if makes the code clearer / faster / more maintainable (but open issues against them, if you can).

icemac

Thank you for your hard work. I do not know enough C to be able to review the changes.

icemac · 2016-11-08T10:34:17Z

tox.ini

@@ -1,6 +1,6 @@
 [tox]
 envlist =
-    py27,py27-pure,py33,py34,py35,pypy,pypy3,coverage
+    py27,py27-pure,py33,py34,py34-pure,py35,pypy,pypy3,coverage


Is there a reason to test this specific Python 3 version as pure python?

There is no specific reason why I have chosen this version. My goal was that at least one python3 version tests the python only implementation.
I think for completeness I would make sense to add a also a py33-pure and py35-pure possibly 3.6. I'll update the pull request in the next days.

Background:
Per default a test uses the C-Extension. In a pure environment the C-Extension is not compiled nor considered for the unittests. So only the pure environments test the python only implementation of ExtensionClass. Which means with the current envlist the python only implementation is tested in python27 and python34.

Merge pull request #9 from axiros/ExtensionClass/master.

hannosch · 2017-01-18T16:29:08Z

I've reviewed this to the best of my ability and found no issues.

I've squashed and merged this manually in f6ef36a. In subsequent commits I've extended the tox test list of cover all combinations of pure/non-pure environments and all Python versions addressing icemac's comment.

tseaver · 2017-01-18T16:33:20Z

@hannosch Thanks for picking up my slack!

@stephan-hof Thanks again for your patch!

hannosch · 2017-01-18T16:35:25Z

@tseaver you are welcome :) @stephan-hof if you want to tackle that Acquisition C port now ... ;)

stephan-hof · 2017-01-19T08:37:37Z

@hannosch many thanks for reviewing it.

stephan-hof added 21 commits September 30, 2016 18:00

Make the unittest working with the python2.7 C-Extension.

d36ff06

Rewrite Base_getattro

519ff5f

Basically do what python version does. Call object.__getattribute__ and if it's a descriptor for something besides __parent__, call it.

Drop support for python < 2.6.

b4182a6

Py_TPFLAGS_HAVE_VERSION_TAG is available since python2.6

Beautify the class defitions.

9ad3ac2

Use PyVarObject_HEAD_INIT.

18ee018

This macro works for python2 and python3.

Rewrite inheritedAttribute

d099074

Basically do the same as the python version. Get the super class and search for the attribute there.

Copy pickle support from 'zopefoundation/persistent'

2731726

pickle support in persistent is already python3 ready so reuse the implementation. The python version is also a copy of persistent and the previous C version look also like a copy&paste from persistent.

Improve the method check in EC_init.

736a408

In python3 unbound method disappeared. So it is very likely that __class_init__ is just a function. => Allow functions also as __class_init__

Make ob_type and 'to python strint from char*' compatible with py2 an…

18c2dd7

…d py3.

Port the module initialization.

f427a86

Port the header file.

bb45918

A relative import is needed in python3.

3258a95

Otherwise the import of python3 (which is absolute now), does not pick up the C-Extension.

Port _ComputedAttribute

99e556e

Port _MethodObject

49414ad

Change setup.py to compile the C-Extensions for python3 too.

33cbeb9

Add CHANGES.rst to MANIFEST.in otherwise its not added to the sdist.

e463139

Test the C-Extension and the python version in py3 too.

c8717f4

Refactor the name check of setattro into a separate function.

a86555a

This makes it easier to Py_DECREF(as_bytes) in case of errors.

Cleanup _compat.h

37da228

icemac reviewed Nov 8, 2016

View reviewed changes

hannosch pushed a commit that referenced this pull request Jan 18, 2017

Port the C extensions to Python 3.

f6ef36a

Merge pull request #9 from axiros/ExtensionClass/master.

hannosch closed this Jan 18, 2017

hannosch mentioned this pull request Jan 18, 2017

Port the C extension to Python 3? #7

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python3 port of the C-Extension #9

Python3 port of the C-Extension #9

stephan-hof commented Sep 30, 2016

tseaver commented Oct 1, 2016

tseaver commented Oct 2, 2016

stephan-hof commented Oct 4, 2016

jamadden commented Oct 4, 2016

tseaver commented Oct 4, 2016

icemac left a comment

icemac Nov 8, 2016

stephan-hof Nov 8, 2016

hannosch commented Jan 18, 2017

tseaver commented Jan 18, 2017

hannosch commented Jan 18, 2017

stephan-hof commented Jan 19, 2017

Python3 port of the C-Extension #9

Python3 port of the C-Extension #9

Conversation

stephan-hof commented Sep 30, 2016

tseaver commented Oct 1, 2016

tseaver commented Oct 2, 2016

stephan-hof commented Oct 4, 2016

jamadden commented Oct 4, 2016

tseaver commented Oct 4, 2016

icemac left a comment

Choose a reason for hiding this comment

icemac Nov 8, 2016

Choose a reason for hiding this comment

stephan-hof Nov 8, 2016

Choose a reason for hiding this comment

hannosch commented Jan 18, 2017

tseaver commented Jan 18, 2017

hannosch commented Jan 18, 2017

stephan-hof commented Jan 19, 2017