forked from cython/cython
-
Notifications
You must be signed in to change notification settings - Fork 0
/
extension_types.html
444 lines (441 loc) · 24 KB
/
extension_types.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
<!DOCTYPE doctype PUBLIC "-//w3c//dtd html 4.0 transitional//en">
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Mozilla/4.61 (Macintosh; I; PPC) [Netscape]"><title>Extension Types</title></head>
<body>
<h1> <hr width="100%">Extension Types
<hr width="100%"></h1>
<h2> Contents</h2>
<ul>
<li> <a href="#Introduction">Introduction</a></li>
<li> <a href="#ExtTypeAttrs">Attributes</a></li>
<li> <a href="#NotNone">Extension types and None</a></li>
<li> <a href="special_methods.html">Special methods</a></li>
<li> <a href="#Properties">Properties</a> <font style="color: rgb(0, 153, 0);" color="#ed181e">(NEW in
0.9)</font></li>
<li><a href="#SubclassingExtTypes">Subclassing</a></li>
<li> <a href="#CMethods">C Methods</a> <font style="color: rgb(0, 153, 0);" color="#ff0000">(NEW in 0.9)</font><br>
<a href="#ForwardDeclaringExtTypes">Forward-declaring extension types</a></li><li><a href="#WeakRefs">Making extension types weak-referenceable</a> <span style="color: rgb(255, 0, 0);">(NEW in 0.9.4)</span><br>
</li>
<li> <a href="#PublicAndExtern">Public and external extension types</a><font color="#2f8b20"><br>
</font></li>
<ul>
<li> <a href="#ExternalExtTypes">External extension types</a></li>
<li> <a href="#ImplicitImport">Implicit importing</a><font color="#2f8b20"><br>
</font></li>
<li> <a href="#TypeVsConstructor">Type names vs. constructor names</a></li>
<li> <a href="#PublicExtensionTypes">Public extension types</a></li>
<li> <a href="#NameSpecClause">Name specification clause</a></li>
</ul>
</ul>
<h2> <a name="Introduction"></a>Introduction</h2>
As well as creating normal user-defined classes with the Python <b>class</b>
statement, Cython also lets you create new built-in Python types, known as
<i>extension types</i>. You define an extension type using the <b>cdef class</b> statement. Here's an example:
<blockquote><tt>cdef class Shrubbery:</tt> <p><tt> cdef int width, height</tt> </p>
<p><tt> def __init__(self, w, h):</tt> <br>
<tt> self.width = w</tt> <br>
<tt> self.height = h</tt> </p>
<p><tt> def describe(self):</tt> <br>
<tt> print "This shrubbery is",
self.width, \</tt> <br>
<tt>
"by", self.height, "cubits."</tt></p>
</blockquote>
As you can see, a Cython extension type definition looks a lot like a Python
class definition. Within it, you use the <b>def</b> statement to define
methods that can be called from Python code. You can even define many of
the special methods such as <tt>__init__</tt> as you would in Python.
<p>The main difference is that you can use the <b>cdef</b> statement to define
attributes. The attributes may be Python objects (either generic or of a particular
extension type), or they may be of any C data type. So you can use extension
types to wrap arbitrary C data structures and provide a Python-like interface
to them. </p>
<h2> <a name="ExtTypeAttrs"></a>Attributes</h2>
Attributes of an extension type are stored directly in the object's C struct.
The set of attributes is fixed at compile time; you can't add attributes
to an extension type instance at run time simply by assigning to them, as
you could with a Python class instance. (You can subclass the extension type
in Python and add attributes to instances of the subclass, however.)
<p>There are two ways that attributes of an extension type can be accessed:
by Python attribute lookup, or by direct access to the C struct from Cython
code. Python code is only able to access attributes of an extension type
by the first method, but Cython code can use either method. </p>
<p>By default, extension type attributes are only accessible by direct access,
not Python access, which means that they are not accessible from Python code.
To make them accessible from Python code, you need to declare them as <tt>public</tt> or <tt>readonly</tt>. For example, </p>
<blockquote><tt>cdef class Shrubbery:</tt> <br>
<tt> cdef public int width, height</tt> <br>
<tt> cdef readonly float depth</tt></blockquote>
makes the <tt>width</tt> and <tt>height</tt> attributes readable and writable
from Python code, and the <tt>depth</tt> attribute readable but not writable.
<p>Note that you can only expose simple C types, such as ints, floats and
strings, for Python access. You can also expose Python-valued attributes,
although read-write exposure is only possible for generic Python attributes
(of type <tt>object</tt>). If the attribute is declared to be of an extension
type, it must be exposed <tt>readonly</tt>. </p>
<p>Note also that the <tt>public</tt> and <tt>readonly</tt> options apply
only to <i>Python</i> access, not direct access. All the attributes of an
extension type are always readable and writable by direct access. </p>
<p>Howerver, for direct access to be possible, the Cython compiler must know
that you have an instance of that type, and not just a generic Python object.
It knows this already in the case of the "self" parameter of the methods of
that type, but in other cases you will have to tell it by means of a declaration.
For example, </p>
<blockquote><tt>cdef widen_shrubbery(Shrubbery sh, extra_width):</tt> <br>
<tt> sh.width = sh.width + extra_width</tt></blockquote>
If you attempt to access an extension type attribute through a generic
object reference, Cython will use a Python attribute lookup. If the attribute
is exposed for Python access (using <tt>public</tt> or <tt>readonly</tt>)
then this will work, but it will be much slower than direct access.
<h2> <a name="NotNone"></a>Extension types and None</h2>
When you declare a parameter or C variable as being of an extension type,
Cython will allow it to take on the value None as well as values of its declared
type. This is analogous to the way a C pointer can take on the value NULL,
and you need to exercise the same caution because of it. There is no problem
as long as you are performing Python operations on it, because full dynamic
type checking will be applied. However, when you access C attributes of an
extension type (as in the <tt>widen_shrubbery</tt> function above), it's up
to you to make sure the reference you're using is not None -- in the interests
of efficiency, Cython does <i>not</i> check this.
<p>You need to be particularly careful when exposing Python functions which
take extension types as arguments. If we wanted to make <tt>widen_shrubbery</tt>
a Python function, for example, if we simply wrote </p>
<blockquote><tt>def widen_shrubbery(Shrubbery sh, extra_width): # <font color="#ed181e">This is</font></tt> <br>
<tt> sh.width = sh.width + extra_width
# <font color="#ed181e">dangerous!</font></tt></blockquote>
then users of our module could crash it by passing None for the <tt>sh</tt>
parameter.
<p>One way to fix this would be </p>
<blockquote><tt>def widen_shrubbery(Shrubbery sh, extra_width):</tt> <br>
<tt> if sh is None:</tt> <br>
<tt> raise TypeError</tt> <br>
<tt> sh.width = sh.width + extra_width</tt></blockquote>
but since this is anticipated to be such a frequent requirement, Cython
provides a more convenient way. Parameters of a Python function declared
as an extension type can have a <b><tt>not None</tt></b> clause:
<blockquote><tt>def widen_shrubbery(Shrubbery sh not None, extra_width):</tt>
<br>
<tt> sh.width = sh.width + extra_width</tt></blockquote>
Now the function will automatically check that <tt>sh</tt> is not None
along with checking that it has the right type.
<p>Note, however that the <tt>not None</tt> clause can <i>only</i> be used
in Python functions (defined with <tt>def</tt>) and not C functions (defined
with <tt>cdef</tt>). If you need to check whether a parameter to a C function
is None, you will need to do it yourself. </p>
<p>Some more things to note: </p>
<ul>
<li> The <b>self</b> parameter of a method of an extension type is guaranteed
never to be None.</li>
</ul>
<ul>
<li> When comparing a value with None, keep in mind that, if <tt>x</tt> is a Python object, <tt>x is None</tt> and <tt>x is not None</tt> are very
efficient because they translate directly to C pointer comparisons, whereas
<tt>x == None</tt> and <tt>x != None</tt>, or simply using <tt>x</tt> as a boolean value (as in <tt>if x: ...</tt>) will invoke Python operations
and therefore be much slower.</li>
</ul>
<h2> <a name="ExtTypeSpecialMethods"></a>Special methods</h2>
Although the principles are similar, there are substantial differences
between many of the <span style="font-family: monospace;">__xxx__</span> special methods of extension types and their
Python counterparts. There is a <a href="special_methods.html">separate page</a> devoted to this subject, and you should read it carefully before attempting
to use any special methods in your extension types.
<h2> <a name="Properties"></a>Properties</h2>
There is a special syntax for defining <b>properties</b> in an extension
class:
<blockquote><tt>cdef class Spam:</tt> <p><tt> property cheese:</tt> </p>
<p><tt> "A doc string can go
here."</tt> </p>
<p><tt> def __get__(self):</tt>
<br>
<tt>
# This is called when the property is read.</tt> <br>
<tt>
...</tt> </p>
<p><tt> def __set__(self, value):</tt>
<br>
<tt>
# This is called when the property is written.</tt> <br>
<tt>
...</tt> </p>
<p><tt> def __del__(self):</tt>
<br>
<tt>
# This is called when the property is deleted.</tt> <br>
</p>
</blockquote>
The <tt>__get__</tt>, <tt>__set__</tt> and <tt>__del__</tt> methods are
all optional; if they are omitted, an exception will be raised when the corresponding
operation is attempted.
<p>Here's a complete example. It defines a property which adds to a list
each time it is written to, returns the list when it is read, and empties
the list when it is deleted. <br>
</p>
<center> <table align="center" cellpadding="5">
<tbody>
<tr>
<td bgcolor="#ffaf18"><b><tt>cheesy.pyx</tt></b></td>
<td bgcolor="#5dbaca"><b>Test input</b></td>
</tr>
<tr>
<td rowspan="3" bgcolor="#ffaf18" valign="top"><tt>cdef class CheeseShop:</tt>
<p><tt> cdef object cheeses</tt> </p>
<p><tt> def __new__(self):</tt> <br>
<tt> self.cheeses = []</tt> </p>
<p><tt> property cheese:</tt> </p>
<p><tt> def __get__(self):</tt> <br>
<tt> return "We don't have: %s" % self.cheeses</tt>
</p>
<p><tt> def __set__(self, value):</tt> <br>
<tt> self.cheeses.append(value)</tt>
</p>
<p><tt> def __del__(self):</tt> <br>
<tt> del self.cheeses[:]</tt></p>
</td>
<td bgcolor="#5dbaca" valign="top"><tt>from cheesy import CheeseShop</tt>
<p><tt>shop = CheeseShop()</tt> <br>
<tt>print shop.cheese</tt> </p>
<p><tt>shop.cheese = "camembert"</tt> <br>
<tt>print shop.cheese</tt> </p>
<p><tt>shop.cheese = "cheddar"</tt> <br>
<tt>print shop.cheese</tt> </p>
<p><tt>del shop.cheese</tt> <br>
<tt>print shop.cheese</tt></p>
</td>
</tr>
<tr>
<td bgcolor="#8cbc1c"><b>Test output</b></td>
</tr>
<tr>
<td bgcolor="#8cbc1c"><tt>We don't have: []</tt> <br>
<tt>We don't have: ['camembert']</tt> <br>
<tt>We don't have: ['camembert', 'cheddar']</tt> <br>
<tt>We don't have: []</tt></td>
</tr>
</tbody> </table>
</center>
<h2> <a name="SubclassingExtTypes"></a>Subclassing</h2>
An extension type may inherit from a built-in type or another extension
type:
<blockquote><tt>cdef class Parrot:</tt> <br>
<tt> ...</tt><tt></tt> <p><tt>cdef class Norwegian(Parrot):</tt> <br>
<tt> ...</tt></p>
</blockquote>
<p><br>
A complete definition of the base type must be available to Cython, so if
the base type is a built-in type, it must have been previously declared as
an <b>extern</b> extension type. If the base type is defined in another Cython
module, it must either be declared as an extern extension type or imported
using the <b><a href="sharing.html">cimport</a></b> statement. </p>
<p>An extension type can only have one base class (no multiple inheritance).
</p>
<p>Cython extension types can also be subclassed in Python. A Python class
can inherit from multiple extension types provided that the usual Python
rules for multiple inheritance are followed (i.e. the C layouts of all the
base classes must be compatible).<br>
</p>
<h2><a name="CMethods"></a>C methods</h2>
Extension types can have C methods as well as Python methods. Like C functions,
C methods are declared using <tt>cdef</tt> instead of <tt>def</tt>. C methods
are "virtual", and may be overridden in derived extension types.<br>
<br>
<table align="center" cellpadding="5">
<tbody>
<tr>
<td bgcolor="#ffaf18" valign="top" width="50%"><b><tt>pets.pyx</tt></b><br>
</td>
<td bgcolor="#8cbc1c" valign="top" width="30%"><b>Output</b><br>
</td>
</tr>
<tr>
<td bgcolor="#ffaf18" valign="top" width="50%"><tt>cdef class Parrot:<br>
<br>
cdef void describe(self):<br>
print "This parrot is resting."<br>
<br>
cdef class Norwegian(Parrot):<br>
<br>
cdef void describe(self):<br>
Parrot.describe(self)<br>
print "Lovely plumage!"<br>
<br>
<br>
cdef Parrot p1, p2<br>
p1 = Parrot()<br>
p2 = Norwegian()<br>
print "p1:"<br>
p1.describe()<br>
print "p2:"<br>
p2.describe()</tt> <br>
</td>
<td bgcolor="#8cbc1c" valign="top" width="30%"><tt>p1:<br>
This parrot is resting.<br>
p2:<br>
</tt><tt>This parrot is resting.<br>
</tt><tt> Lovely plumage!</tt><br>
</td>
</tr>
</tbody> </table>
<br>
The above example also illustrates that a C method can call an inherited
C method using the usual Python technique, i.e.<br>
<blockquote><tt>Parrot.describe(self)</tt><br>
</blockquote>
<h2><a name="ForwardDeclaringExtTypes"></a>Forward-declaring extension types</h2>
Extension types can be forward-declared, like struct and union types. This
will be necessary if you have two extension types that need to refer to
each other, e.g.
<blockquote><tt>cdef class Shrubbery # forward declaration</tt> <p><tt>cdef class Shrubber:</tt> <br>
<tt> cdef Shrubbery work_in_progress</tt> </p>
<p><tt>cdef class Shrubbery:</tt> <br>
<tt> cdef Shrubber creator</tt></p>
</blockquote>
If you are forward-declaring an exension type that has a base class, you
must specify the base class in both the forward declaration and its subsequent
definition, for example,<br>
<blockquote><tt>cdef class A(B)<br>
<br>
...<br>
<br>
cdef class A(B):<br>
# attributes and methods</tt><br>
</blockquote>
<h2><a name="WeakRefs"></a>Making extension types weak-referenceable</h2>By
default, extension types do not support having weak references made to
them. You can enable weak referencing by declaring a C attribute of
type <span style="font-family: monospace;">object</span> called <span style="font-family: monospace; font-weight: bold;">__weakref__</span>. For example,<br>
<br>
<div style="margin-left: 40px;"><span style="font-family: monospace;">cdef class ExplodingAnimal:</span><br style="font-family: monospace;">
<span style="font-family: monospace;"> """This animal will self-destruct when it is</span><br>
<span style="font-family: monospace;"> no longer strongly referenced."""</span><br>
<span style="font-family: monospace;"> </span><br style="font-family: monospace;">
<span style="font-family: monospace;"></span><span style="font-family: monospace;"> cdef object __weakref__</span><br>
</div>
<br>
<h2><a name="PublicAndExtern"></a>Public and external extension types</h2>
Extension types can be declared <b>extern</b> or <b>public</b>. An <a href="#ExternalExtTypes"><b>extern</b> extension type declaration</a> makes
an extension type defined in external C code available to a Cython module.
A <a href="#PublicExtensionTypes"><b>public</b> extension type declaration</a> makes an extension type defined in a Cython module available to external C
code.
<h3> <a name="ExternalExtTypes"></a>External extension types</h3>
An <b>extern</b> extension type allows you to gain access to the internals
of Python objects defined in the Python core or in a non-Cython extension
module.
<blockquote><b>NOTE:</b> In Cython versions before 0.8, <b>extern</b> extension
types were also used to reference extension types defined in another Cython
module. While you can still do that, Cython 0.8 and later provides a better
mechanism for this. See <a href="sharing.html">Sharing C Declarations Between
Cython Modules</a>.</blockquote>
Here is an example which will let you get at the C-level members of the
built-in <i>complex</i> object.
<blockquote><tt>cdef extern from "complexobject.h":</tt> <p><tt> struct Py_complex:</tt> <br>
<tt> double real</tt> <br>
<tt> double imag</tt> </p>
<p><tt> ctypedef class __builtin__.complex [object PyComplexObject]:</tt>
<br>
<tt> cdef Py_complex cval</tt>
</p>
<p><tt># A function which uses the above type</tt> <br>
<tt>def spam(complex c):</tt> <br>
<tt> print "Real:", c.cval.real</tt> <br>
<tt> print "Imag:", c.cval.imag</tt></p>
</blockquote>
Some important things to note are:
<ol>
<li> In this example, <b>ctypedef class</b> has been used. This is because,
in the Python header files, the <tt>PyComplexObject</tt> struct is declared
with<br>
<br>
<div style="margin-left: 40px;"><tt>ctypedef struct {</tt> <br>
<tt> ...</tt> <br>
<tt>} PyComplexObject;<br>
<br>
</tt></div>
</li><li>As well as the name of the extension type, the <i>module</i> in which
its type object can be found is also specified. See the <a href="#ImplicitImport">implicit importing</a> section below. <br>
<br>
</li>
<li> When declaring an external extension type, you don't declare
any methods. Declaration of methods is not required in order to call them,
because the calls are Python method calls. Also, as with structs and unions,
if your extension class declaration is inside a <i>cdef extern from</i> block,
you only need to declare those C members which you wish to access.</li>
</ol>
<h3> <a name="ImplicitImport"></a>Implicit importing</h3>
<blockquote><font color="#ef1f1d">Backwards Incompatibility Note</font>:
You will have to update any pre-0.8 Cython modules you have which use <b>extern</b>
extension types. I apologise for this, but for complicated reasons it proved
to be too difficult to continue supporting the old way of doing these while
introducing the new features that I wanted.</blockquote>
Cython 0.8 and later requires you to include a module name in an extern
extension class declaration, for example,
<blockquote><tt>cdef extern class MyModule.Spam:</tt> <br>
<tt> ...</tt></blockquote>
The type object will be implicitly imported from the specified module and
bound to the corresponding name in this module. In other words, in this
example an implicit
<ol>
<pre>from <tt>MyModule</tt> import Spam</pre>
</ol>
statement will be executed at module load time.
<p>The module name can be a dotted name to refer to a module inside a package
hierarchy, for example, </p>
<blockquote><tt>cdef extern class My.Nested.Package.Spam:</tt> <br>
<tt> ...</tt></blockquote>
You can also specify an alternative name under which to import the type
using an <b>as</b> clause, for example,
<ol>
<tt>cdef extern class My.Nested.Package.Spam as Yummy:</tt> <br>
<tt> ...</tt> </ol>
which corresponds to the implicit import statement
<ol>
<pre>from <tt>My.Nested.Package</tt> import <tt>Spam</tt> as <tt>Yummy</tt></pre>
</ol>
<h3> <a name="TypeVsConstructor"></a>Type names vs. constructor names</h3>
Inside a Cython module, the name of an extension type serves two distinct
purposes. When used in an expression, it refers to a module-level global
variable holding the type's constructor (i.e. its type-object). However,
it can also be used as a C type name to declare variables, arguments and
return values of that type.
<p>When you declare </p>
<blockquote><tt>cdef extern class MyModule.Spam:</tt> <br>
<tt> ...</tt></blockquote>
the name <tt>Spam</tt> serves both these roles. There may be other names
by which you can refer to the constructor, but only <tt>Spam</tt> can be
used as a type name. For example, if you were to explicity <tt>import MyModule</tt>,
you could use<tt> MyModule.Spam()</tt> to create a Spam instance, but you
wouldn't be able to use <tt>MyModule.Spam</tt> as a type name.
<p>When an <b>as</b> clause is used, the name specified in the <b>as</b>
clause also takes over both roles. So if you declare </p>
<blockquote><tt>cdef extern class MyModule.Spam as Yummy:</tt> <br>
<tt> ...</tt></blockquote>
then <tt>Yummy</tt> becomes both the type name and a name for the constructor.
Again, there are other ways that you could get hold of the constructor,
but only <tt>Yummy</tt> is usable as a type name.
<h3> <a name="PublicExtensionTypes"></a>Public extension types</h3>
An extension type can be declared <b>public</b>, in which case a <b>.h</b>
file is generated containing declarations for its object struct and type
object. By including the <b>.h</b> file in external C code that you write,
that code can access the attributes of the extension type.
<h3> <a name="NameSpecClause"></a>Name specification clause</h3>
The part of the class declaration in square brackets is a special feature
only available for <b>extern</b> or <b>public</b> extension types. The full
form of this clause is
<blockquote><tt>[object </tt><i>object_struct_name</i><tt>, type </tt><i>type_object_name</i><span style="font-family: monospace;"> ]</span></blockquote>
where <i>object_struct_name</i> is the name to assume for the type's C
struct, and <i>type_object_name</i> is the name to assume for the type's
statically declared type object. (The object and type clauses can be written
in either order.)
<p>If the extension type declaration is inside a <b>cdef extern from</b>
block, the <b>object</b> clause is required, because Cython must be able to
generate code that is compatible with the declarations in the header file.
Otherwise, for <b>extern</b> extension types, the <b>object</b> clause is
optional. </p>
<p>For <b>public</b> extension types, the <b>object</b> and <b>type</b> clauses
are both required, because Cython must be able to generate code that is compatible
with external C code. </p>
<p> </p>
<hr width="100%"> <br>
Back to the <a href="overview.html">Language Overview</a> <br>
<br>
<br>
</body></html>