Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Contents

What is the relation between Cython and Pyrex? Are the barriers between the two based on technical direction? Differing goals?

Somewhat. Cython is much more open to extensions than Pyrex. Greg usually says that he's still "designing" Pyrex as a language, so he will sometimes reject patches for design reasons that solve practical problems in a practical way, and that therefore find (or found) their way into Cython. Eventually, these features might still make it into Pyrex in one way or another, but that usually means that Greg refactors or rewrites them his own way, which implies that he first has to find the time to do so.

Cython can afford to be more agile and advanced even though doing so means that Cython will not always align with future Pyrex versions. However, both Greg Ewing and the Cython developers make reasonable effort to maintain compatibility.

Today, Cython is an advanced version of Pyrex that has several additions already integrated that never made it into mainline Pyrex, including:

  1. Conditional expressions (a if blah else b)
  2. List/set/dict comprehensions
  3. Optimized looping (for x in blah: is much faster in Cython)
  4. Compatibility with Python 3 (as well as Python 2.4 or later) without regenerating the C code
  5. Support for the new buffer protocol (PEP 3118), featuring efficient access to data structures in NumPy or PIL

The intention is to make it for the most part a drop-in replacement for existing Pyrex code, though some changes to that existing code may have to be made. The immediate speed-up is generally worth the switch.

To you as a user this means that if you use Cython today, you can write your code a lot cleaner and simpler now as you can rely on Cython to optimise it for you in a lot of ways that you do not have to care about. But if you use Cython specific syntax features (i.e. syntax elements that are not described in the documentation of Pyrex or Python), you may have to do minor syntactic code changes in the near or far future if you want to go back to a future Pyrex version. In general, however, both Pyrex and Cython try to adhere to the existing Python syntax as close as possible, so these cases should be rare.

In early versions, Cython used to follow a 4-digit versioning scheme that kept the corresponding Pyrex version in the first three digits. As most of the development in Cython is now completely independent from what is going on with Pyrex, we have broken with this scheme. Cython versions are now unrelated to Pyrex versions.


Is Cython a Python implementation?

Not officially, no. However, it compiles almost all existing Python code, which gets it pretty close to a real Python implementation. In any case, it is an official goal for Cython 1.0 to compile regular Python code and run (most of) the normal Python test suite - obviously faster than CPython. ;-)


Is Cython faster than CPython?

For most things, yes. For example, Cython can compile most of pybench by now, and runs it more than 30% faster in total, while being 60-90% faster on control structures like if-elif-else and for-loops. We regularly run the tests from the CPython benchmark suite and many of them work out-of-the-box without modifications. See the latest results.

However the main advantage of Cython is that it scales very well to even greater performance requirements. For numerical code, speed-ups of 100-1000 times compared to CPython are not unusual, and are achieved by simply adding static type declarations to performance critical parts of the code, thus trading Python's dynamic typing for speed. As this can be done at any granularity in the code, Cython makes it easy to write simple Python code that is fast enough, and just tune the critical 5% of your code into maximum performance by using static C types in just the right places.


What Python versions does Cython support?

Version 0.20 of the Cython compiler runs in all Python versions from 2.4 to 3.4 inclusive (excluding 3.0). From Cython 0.21 on, support for CPython 2.4, 2.5 and 3.1 has been dropped, so that the supported versions become 2.6, 2.7, 3.2 and later.

The C code generated by Cython is portable and builds in all CPython versions from 2.6, 2.7 and 3.2 upwards (2.4 through 3.4 for Cython 0.20). All supported CPython release series are tested regularly. New CPython versions are usually supported before they are released.

The source code that Cython compiles can use both Python 2 and Python 3 syntax. When compiling Cython modules (.pyx files), most Python 3 syntax features are available by default if they do not interfere with Python 2 syntax (as in Python 2.7), but the general language semantics are defined as in Python 2. When compiling Python modules (.py files), the language level is Python 2 by default and the special Cython syntax (such as the cdef keyword) is not available. For both input types, the language level can be set to Python 3 by either passing the "-3" option to the compiler, or by putting

# cython: language_level=3

at the top of the module file (within the first comment and before any code or empty lines). The most visible difference is that unprefixed strings are unicode strings under Python 3 semantics, but this also makes print() a function, keeps loop variables in list comprehensions from leaking into the outer scope, etc.


Can Cython generate C code for classes?

Answer: Yes, these classes become fully fledged Python classes.


How do I pickle cdef classes?

Answer: You have to implement the pickle protocol. Automatic pickle support (at least for attributes with Python compatible types) is still pending.


Is it possible to make a cdef'd class that derives from a builtin Python type such as list?

Answer: Yes, since Cython 0.14, you can just use the type as a base class in your cdef class declaration. Older versions of Cython required a work-around that has several drawbacks, especially for optimisations.

The only exception are the types bytes ('str' in Python 2) and tuple, which can only be subtyped by Python classes (not cdef classes). This is considered a bug. However, you can safely subtype 'unicode' and 'list' instead.


Can I place the output under the BSD license, or does it have to be the python-license as well?

Answer: You can use the output of Pyrex/Cython however you like (and license it how you like - be it BSD, public domain, GPL, all rights reserved, whatever).

More details: The Python License is different from the GPL used for GCC, for example. GCC requires a special exception clause for its output as it is linked against the library part of GCC, i.e. against GPL software, which triggers the GPL restrictions.

Pyrex doesn't do anything similar, and linking against Python is not restricted by the Python License, so the output belongs to the User, no other rights or restrictions involved.

Also, all of the copyright holders of Pyrex/Cython stated in mailing list that people are allowed to use the output of Pyrex/Cython however they would like.


How do I cite Cython in an academic paper?

If you mention Cython, the simplest way to reference us is to add the URL to our website in a footnote. You may also choose to reference our software project in a more formal way, such as

R. Bradshaw, S. Behnel, D. S. Seljebotn, G. Ewing, et al., The Cython compiler, http://cython.org.

(the list of author names were taken from setup.py)

For a yet more formal citation, there is a journal paper on Cython; unfortunately it is behind the academic paywall (although preprints are freely accessible). If you wish to cite it, here's the Bibtex:

@ARTICLE{ behnel2010cython,
    author={Behnel, S. and Bradshaw, R. and Citro, C. and Dalcin, L. and Seljebotn, D.S. and Smith, K.},
    journal={Computing in Science Engineering},
    title={Cython: The Best of Both Worlds},
    year={2011},
    month=march-april ,
    volume={13},
    number={2},
    pages={31 -39},
    keywords={Cython language;Fortran code;Python language extension;numerical loops;programming language;C language;numerical analysis;},
    doi={10.1109/MCSE.2010.118},
    ISSN={1521-9615},
}

''Why does ** on int literals not work (as it seems to do in Pyrex)?''

It works as expected in recent versions of Cython.

In older versions, it was considered that the fact that a binary operation on two integer types returned a float was counter-intuitive (both compared to every other kind of binary op in C, and the "expected" behavior from python). We discovered it because it was causing errors (e.g. in functions that were expecting an integer value but getting a float) and after much discussion decided that disabling this behavior was better than letting it go. Also a**b will (silently) overflow as an int/be inexact as a double except for very small values of b. If one wants the old behavior, one can always do, e.g, 13.0**5, where it is much clearer what's going on. One would have to do <int>(13**5) in pyrex anyway, which looks kind of strange.


How to pass string buffers that may contain 0 bytes to Cython?

See the string tutorial.

You need to use either a Python byte string object or a char*/length pair of variables.

The normal way to convert a char* to a Python byte string is as follows:

#!python
cdef char* s = "a normal C byte string"
cdef bytes a_python_byte_string = s

However, this will not work for C strings that contain 0 bytes, as a 0 byte is the normal C way of terminating a string. So the above method will cut the string at the first 0 byte. To handle this case correctly, you have to specify the total length of the string that you want to convert:

cdef char* s = "an unusual \0 containing C byte string"
a_python_byte_string = s[:21]    #  take the first 21 bytes of the string, including the \0 byte

Note that this will not handle the case that the specified slice length is longer than the actual C string. This code will crash if the allocated memory area of the char* is shorter.

Since Cython 0.12, there is also support for decoding a C string slice efficiently into a Python unicode string. Just do this:

# -*- coding: ISO8859-15
cdef char* s = "a UTF-8 encoded C string with fünny chäräctörs"
cdef Py_ssize_t byte_length = 46

a_python_unicode_string = s[:byte_length].decode('ISO8859-15')

Can Cython create objects or apply operators to locally created objects as pure C code?

For methods like __init__ and __getitem__ the Python calling convention is mandatory and identical for all objects, so Cython cannot provide a major speed-up for them.

To instantiate an extension type in Cython 0.12, however, the fastest way is to actually use the normal Python idiom of calling the __new__() method of a type:

cdef class ExampleClass:
    cdef int _value
    def __init__(self):
        # calling "__new__()" will not call "__init__()" !
        raise TypeError("This class cannot be instantiated from Python")

cdef ExampleClass _factory():
    cdef ExampleClass instance = ExampleClass.__new__(ExampleClass)
    instance._value = 1
    return instance

Note that this has similar restrictions as the normal Python code: it will not call the __init__() method (which makes it quite a bit faster). Also, while all Python class members will be initialised to None, you have to take care to initialise the C members. Either the __cinit__() method or a factory function like the one above are good places to do so.

In Cython 0.11 and older versions, you had to use the following C-ish hack in an external header file:

/* in FILE "theheader.h" */
#define PY_NEW(T) \
     (((PyTypeObject*)(T))->tp_new( \
             (PyTypeObject*)(T), __pyx_empty_tuple, NULL))

and then define it as a Cython function as follows:

cdef extern from "theheader.h":
    # macro call to 't->tp_new()' for fast instantiation
    cdef ExampleClass NEW_EXAMPLE_CLASS "PY_NEW" (object t)

cdef ExampleClass _factory():
    cdef ExampleClass instance = NEW_EXAMPLE_CLASS(ExampleClass)
    instance._value = 1
    return instance

Why does Cython not always give errors for uninitialized variables?

Answer: Cython does some static checks for variable initialization before use during compile time, but these are very basic, as Cython has no definite knowledge what paths of code will be taken at runtime:

Consider the following

def testUnboundedLocal1():
   if False:
      c = 1
   print c
def testUnboundedLocal2():
   print c

With CPython, both functions lead to the following exception:

NameError: global name 'c' is not defined

With Cython, the first variant prints "None", the second variant leads to a compile time error. Both behaviours differ from CPython's.

This is considered a BUG and will change in the future.


How well is Unicode supported?

Answer: The support for Unicode is as good as CPythons, as long as you are using the Python unicode string type. But there is no equivalent C type available for Unicode strings. To prevent user errors, Cython will also disallow any implicit conversion to char* as this not going to be correct.

Since Cython 0.13, there is also native support for the Py_UNICODE type that represents a single unicode character. In fact, Cython will try to infer this type for single character unicode literals, and avoid the creation of a unicode string object for them if possible. This is because many operations work much more efficiently (in plain C) on Py_UNICODE than on unicode objects.

See the string tutorial.


How do I pass a Python string parameter on to a C library?

See the string tutorial.

Answer: It depends on the semantics of the string. Imagine you have this C function:

cdef extern from "something.h":
    cdef int c_handle_data(char* data, int length)

For __binary data__, you can simply require byte strings at the API level, so that this will work:

def work_with_binary_data(bytes binary_data):
    c_handle_data(binary_data, len(binary_data))

It will raise an error (with a message that may or may not be appropriate for your use case) if users pass other things than a byte string.

For __textual data__, however, you must handle Unicode data input. What you do with it depends on what your C function accepts. For example, if it requires UTF-8 encoded byte sequences, this might work:

def work_with_text_data(text):
    if not isinstance(text, unicode):
        raise ValueError("requires text input, got %s" % type(text))
    utf8_data = text.encode('UTF-8')
    c_handle_data( utf8_data, len(utf8_data) )

Note that this also accepts subtypes of the Python unicode type. Typing the "text" parameter as "unicode" will not cover this case.

The above is the right thing to do in Py3. However, some (not all, just some) module APIs may become more user friendly in Python 2.x if you additionally allow well defined byte strings. For example, it may make sense to allow plain ASCII strings in some cases, as they are often used for textual data in Python 2.x programs. This could be done as follows:

from python_version cimport PY_MAJOR_VERSION

def work_with_text_data(text):
    if isinstance(text, unicode): # most common case first
        utf8_data = text.encode('UTF-8')
    elif (PY_MAJOR_VERSION < 3) and isinstance(text, str):
        text.decode('ASCII') # trial decoding, or however you want to check for plain ASCII data
        utf8_data = text
    else:
        raise ValueError("requires text input, got %s" % type(text))
    c_handle_data(utf8_data, len(utf8_data))

Can I use builtins like len() with the C type char *?

Answer: Yes you can. Cython 0.12.1 and later map len(char*) directly to strlen(), which means that it will count the number of characters up to the first 0 byte. Similarly, (char*).decode(...) is optimised into a C-API call since 0.12, and applying it to sliced char* values will skip the length counting step.

See the string tutorial.

For other Python operations on char*, the generated code may be inefficient, as a temporary object may have to get created. If you notice this for your code and think that Cython can do better, please speak up on the mailing list.


How do I declare numeric or integer C types?

Answer: In most cases, you don't need to. For types declared in stdint.h, just cimport them from libc.stdint which comes with Cython, e.g.

from libc.stdint cimport uint32_t, int64_t
cdef int64_t i = 5

For non-standard types, it's enough to provide Cython with a ctypedef declaration that maps them to a closely related standard C type, e.g.

cdef extern from "someheader.h":
    ctypedef unsigned long MySpecialCInt_t

cdef MySpecialCInt_t i

Make sure you use the original C type name in declarations, not the replacement type you chose!

The exact size of the type at C compile time is not that important because Cython generates automatic size detection code (evaluated at C compile time). However, when your code mixes different types in arithmetic code, Cython must know about the correct signedness and the approximate longness in order to infer the appropriate result type of an expression. Therefore, when using a ctypedef as above, try to come up with a good approximation of the expected C type. Since the largest type wins in mixed arithmetic expressions, it's usually not a problem if the type turns out to be somewhat larger than what the C compiler eventually determines for a given platform. In the worst case, if your replacement type is substantially larger than the real C type (say, 'long long' instead of 'int'), you may end up with slightly slower conversion code. However, if the type is declared too small and Cython considers it smaller than other types it is used together with, Cython may infer the wrong type for an expression and may end up generating incorrect coercion code. You may or may not get a warning by the C compiler in this case.

Also note that Cython 0.14 and later will consider large integer literals (>32 bit signed) unsafe to use in C code and may therefore use Python objects to represent them. You can make sure a large literal is considered a safe C literal by appending a C suffix, such as 'LL' or 'UL'. Note that a single 'L' is not considered a C suffix in Python 2 code.


How can I interface numpy arrays using Cython?

Answer: Follow the example: http://wiki.cython.org/WrappingNumpy


How can I help cython find numpy header files?

Answer: If you are seeing errors like these:

error: numpy/arrayobject.h: No such file or directory
error: numpy/ufuncobject.h: No such file or directory

You should modify your setup.py file to grab the numpy include directory as follows:

import numpy
...
setup(
    ...
    ext_modules = [Extension(..., include_dirs=[numpy.get_include()])]
)

Is it possible to call my Python code from C?

Answer: Yes, easily. Follow the example in Demos/callback/ in the Cython source distribution.


What is the difference between PyObject* and object?

Answer: A variable of type PyObject* is a simple C pointer, just like void*. It is not reference counted, which is sometimes referred to as a borrowed reference. An object variable is an owned reference to a Python object. You can convert one into the other by casting:

from cpython.ref cimport PyObject

py_object = [1,2,3]

cdef PyObject* ptr = <PyObject*>py_object

cdef object l = <object>ptr    # this increases the reference count to the list

Note that the lifetime of the object is only bound to its owned references, not to any C pointers that happen to point to it. This means that ptr in the example above becomes invalid as soon as the last reference to the object dies:

py_object = [1,2,3]
cdef PyObject* ptr = <PyObject*>py_object
py_object = None   # last reference to list dies here

# ptr now points to a dead object
print(<object>ptr)   # expect a crash here!

Pointers are commonly used when passing objects through C callbacks, e.g.

cdef int call_it_from_c(void* py_function, void* args):
    py_args = <tuple>args if args is not NULL else ()
    return (<object>py_function)(*py_args)

def py_func(a,b,c):
    print(a,b,c)
    return -1

args = [1,2,3]

call_it_from_c(<PyObject*>py_func, <PyObject*>args)

Once again, care must be taken to keep the objects alive as long as any pointers to them are still in use.


What is the difference between a .pxd and .pxi file? When should either be used?

SHORT Answer: You should always use .pxd files for declarations and .pxi files only for code that you want to include.

MEDIUM Answer: A .pxd files are lists of declarations, .pxi files are textually included, and their use for declarations is a historical artifact of the way common declarations were shared before .pxd files existed.

LONG Answer: A .pxd file is a declaration file, and is used to declare classes, methods, etc. in a C extension module, (typically as implemented in a .pyx file of the same name). It can contain declarations only, i.e. no executable statements. One can cimport things from .pxd files just as one would import things in Python. Two separate modules cimporting from the same .pxd file will receive identical objects.

A .pxi file is an include file and is textually included (similar to the C #include directive) and may contain any valid Cython code at the given point in the program. It may contain implementations (e.g. common cdef inline functions) which will be copied into both files. For example, this means that if I have a class A declared in a.pxi, and both b.pyx and c.pyx do include a.pxi then I will have two distinct classes b.A and c.A. Interfaces to C libraries (including the Python/C API) have usually been declared in .pxi files (as they are not associated to a specific module). It is also re-parsed at every invocation.

Now that "cimport *" can be used, there is no reason to use .pxi files for external declarations.


How do I access native Python file objects?

Answer: See this small example of how to access Python file objects:

#!python
# Idiom for accessing Python files.
# First, declare the Python macro to access files:
cdef extern from "Python.h":
    ctypedef struct FILE
    FILE* PyFile_AsFile(object)
    void  fprintf(FILE* f, char* s, char* s)
# Next, enter the builtin file class into the namespace:
cdef extern from "fileobject.h":
    ctypedef class __builtin__.file [object PyFileObject]:
        pass
# Now declare the C function that requires a file:
cdef void c_printSomething(FILE* outFile, char* str):
    fprintf(outFile, "%s", str)
# Now create a class or some other definition that uses the function:
ctypedef class ExampleUsingFile:
    def printSomething(self, file outFile, char* str):
        c_printSomething(PyFile_AsFile(outFile), str)

with simple test:

#!python
import sys
import file_example
x = file_example.ExampleUsingFile()
x.printSomething(sys.stdout, "hello world!\n")

Note: This does no longer work in Python 3, where file objects have no representation at the C-API level.


How do I declare a global variable?

Answer:

global variable

How do I assign to a global variable?

You need to declare the variable to be global (see above) before trying to assign to it. Often this occurs when one has code like

cdef int *data

def foo(n):
    data = malloc(n * sizeof(int))

This will result in an error "Cannot convert 'int *' to Python object." This is because, as in Python, assignment declares a local variable. Instead, you must write

cdef int *data

def foo(n):
    global data
    data = malloc(n * sizeof(int))

See http://docs.python.org/tutorial/classes.html#python-scopes-and-name-spaces for more details.


How do I use 'const'?

Answer: Since Cython 0.18, you can just use it in your code and in your declarations.

Earlier versions of Cython did not support const directly and required the following hack to compile it into the C source code:

cdef extern from *:
    ctypedef char* const_char_ptr "const char*"
cdef public void foo_c(const_char_ptr s):
    print s

This textually replaces the type const_char_ptr by const char* and generates this C code:

__PYX_EXTERN_C  DL_EXPORT(void) foo_c(const char* __pyx_v_s);

Note that the above declarations for the different const char* types are still provided by the libc.string standard declarations for backwards compatibility reasons. A cimport from there will do the right thing in Cython 0.18 and later.


How do I implement a single class method in a Cython module ?

Answer: Cython-defined methods don't bind by default, regardless from where they are referenced. Because of this the following does not work:

#!python
import cython_module

class A(object):
    method = cython_module.optimized_method

method is unbound and trying to call it will result in an error:

#!python
>>> a = A()
>>> a.method()
exceptions.TypeError: optimized_method() takes exactly one argument (0 given)

You have can explicitly create a bound method, either in Python:

#!python
import types
import cython_module

class A(object):
    pass

A.method = types.MethodType(cython_module.optimized_method, None, A)

or by using the cython.binding directive to make the method bind automatically, e.g.

cimport cython
@cython.binding(True)
def optimized_method(self, ...):
    ...

How can I run doctests in Cython code (pyx files)?

Answer:

Recent versions of Cython generate a __test__ dictionary in the module that contains all docstrings of Python visible functions and classes that look like doctests (i.e. that contain >>>). The doctest module will properly pick this up and run the doctests.

Older Cython versions suffer from a problem with doctest because it uses inspect.is_function to check whether something is a function, which fails for Cython functions (which instead answer to inspect.is_builtin).

This module (let's call it "cydoctest") offers a Cython-compatible workaround.

#!python
"""
Cython-compatible wrapper for doctest.testmod().

Usage example, assuming a Cython module mymod.pyx is compiled.
This is run from the command line, passing a command to Python:
python -c "import cydoctest, mymod; cydoctest.testmod(mymod)"

(This still won't let a Cython module run its own doctests
when called with "python mymod.py", but it's pretty close.
Further options can be passed to testmod() as desired, e.g.
verbose=True.)
"""

import doctest
import inspect

def _from_module(module, object):
    """
    Return true if the given object is defined in the given module.
    """
    if module is None:
        return True
    elif inspect.getmodule(object) is not None:
        return module is inspect.getmodule(object)
    elif inspect.isfunction(object):
        return module.__dict__ is object.func_globals
    elif inspect.isclass(object):
        return module.__name__ == object.__module__
    elif hasattr(object, '__module__'):
        return module.__name__ == object.__module__
    elif isinstance(object, property):
        return True # [XX] no way not be sure.
    else:
        raise ValueError("object must be a class or function")

def fix_module_doctest(module):
    """
    Extract docstrings from cython functions, that would be skipped by doctest
    otherwise.
    """
    module.__test__ = {}
    for name in dir(module):
       value = getattr(module, name)
       if inspect.isbuiltin(value) and isinstance(value.__doc__, str) and _from_module(module, value):
           module.__test__[name] = value.__doc__

def testmod(m=None, *args, **kwargs):
    """
    Fix a Cython module's doctests, then call doctest.testmod()

    All other arguments are passed directly to doctest.testmod().
    """
    fix_module_doctest(m)
    doctest.testmod(m, *args, **kwargs)

How to wrap C code that uses the restrict qualifier?

Answer: There currently is no way of doing this directly into C code. Cython does not understand the restrict qualifier. However you can wrap your way around it.

See the following example code:

slurp.h

#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <regex.h>
#include <Python.h>

int th_match(char *, char *);

cslurp.c

#include "slurp.h"

int th_match(char *string, char *pattern) {
  int status;
  regex_t re;
  if(regcomp(&re, pattern, REG_EXTENDED|REG_NOSUB) != 0) { return 0; }
  status = regexec(&re, string, (size_t)0, NULL, 0);
  regfree(&re);
  if(status != 0)
    return 0;
  return 1;
}

slurp.pyx

cdef extern from "slurp.h":
    int th_match(char *st, char *pt)

class Slurp:
    '''
    This is a simple, but optimized PEG (Parser Expression Group) parser.
    It will parse through anything you hand it provided what you hand it
    has a readline() method.

    Example:
        import sys
        from thci.ext import slurp
        o = slurp.Slurp()
        o.register_trigger('^root:.*:.*:.*:.*$', sys.stdout.write)
        o.process(open('/etc/passwd', 'r'))
    '''

    def __init__(self):
        ''' __init__(self) '''
        self.map = {}
        self.idx = 0

    def register_trigger(self, patt=None, cback=None, args=None):
        ''' register_trigger(self, patt=None, cback=None, args=None) '''
        if patt == None or cback == None:
            return False
        if args == None: args = False
        self.map[self.idx] = (patt, cback, args)
        self.idx += 0
        return True

    def process(self, fp=None):
        ''' process(self, fp=None) '''
        if fp == None:
            return False
        while True:
            buf = fp.readline()
            if not buf: break
            for patt, cback, args in self.map.values():
                if th_match(buf, patt) == True:
                    if args == False:
                        cback(buf.strip())
                    else:
                        cback(buf.strip(), args)

This avoids the problems using the restrict qualifiers (Such as are needed with the functions declared in regex.h on FreeBSD [at least 7.X]) by allowing the C compiler to handle things going from C to C, Cython's support for this even using the "const trick" doesn't seem to behave properly (at least as of 0.12). the following commands will generate your compiled module from the above source:

cython -o slurp.c slurp.pyx
cc -shared -I/usr/include -I./ -I/usr/local/include/python2.5 -L/usr/local/lib -lpthread -lpython2.5 cslurp.c slurp.c -o slurp.so

It is also possible to use distutils by adding the file cslurp.c (or your files name) to the list of files to be compiled for the extension.


Why does a function with cdef'd parameters accept None?

Answer: It is a fairly common idiom in Python to use None as a way to mean "no value" or "invalid". This doesn't play well with C, as None is not compatible with any C type. To accommodate for this, the default behavior is for functions with cdefed parameters to also accept None. This behavior was inherited from Pyrex, and while it has been proposed that it be changed, it will likely stay (at least for a while) for backwards capability.

You have three choices for how to handle None in your code:

  1. If you want to consider None invalid input, then you need to write code that checks for it, and raised an appropriate exception.
  2. If you want Cython to raise an exception if None is passed in for an extension type parameter, you can use the not None declaration:
def foo(MyClass val not None): <...>

which is a short-hand for

   def foo(MyClass val):
        if val is None: raise <...>
        <...>

3. You can also put ``#cython: nonecheck=True`` at the top of your file and all access will be checked for None, but it

will slow things down, as it is adding a check on every access, rather that once on function call.


How do I work around the "unable to find vcvarsall.bat" error when using MinGW as the compiler (on Windows)?

Answer: This may be fixed in the latest version of Cython, therefore download and install the latest version.

If this is unsucessful, try the following workarounds.

If no python libraries are imported, define the compiler by adding the following statement:

--compiler=mingw32

Therefore, the line should read:

python pyprog.py build_ext --compiler=mingw32 --inplace
This, however, does not solve the issue when using the pyximport method (see the tutorial). Alternatively, the following patch can be applied.

NOTE: This is untested.

Open the file pyximport/pyxbuild.py and add the four lines marked with "+" at the appropriate place.

diff -r 7fbe931e5ab7 pyximport/pyxbuild.py
--- a/pyximport/pyxbuild.py Wed Sep 16 15:50:00 2009 +0200
+++ b/pyximport/pyxbuild.py Fri Sep 18 12:39:51 2009 -0300
@@ -55,6 +55,11 @@
build = dist.get_command_obj('build')
build.build_base = pyxbuild_dir

+ config_files = dist.find_config_files()
+ try: config_files.remove('setup.cfg')
+ except ValueError: pass
+ dist.parse_config_files(config_files)
+
try:
ok = dist.parse_command_line()
except DistutilsArgError:

Finally, if this does not work, create a file called "pydistutils.cfg" in notepad and give it the contents:

[build_ext]
compiler=mingw32
Save this to the home directory, which can be found by typing at the command prompt:
import os
os.path.expanduser('~')

How do I work around the -Wno-long-double error when installing on OS X

Answer:

This is a known issue in OS X with some Python installs. It has nothing to do with Cython, and you will run on the same trouble every time you want to build an C extension module.

This is the most sane (if not the only) way to fix it:

  1. Enter Python prompt, and type this:
>>> from distutils import sysconfig
>>> sysconfig.get_makefile_filename()

That should output the full path of a 'Makefile'... Open that file with any text editor and remove all occurrences of '-Wno-long-double' flag. ----------

How do I raise an exception in Cython code that will be visible to ancestor (in the callstack) CPython code?

Answer:

If your cdef or cpdef function or method does not declare a return type (as is normal in CPython code), then you get exceptions without any extra effort.

If your cdef or cpdef function or method declares a C-style return type, then look here.


How do I use variable args.

It can't be done cleanly yet, but the code below works:

cdef extern from "stdarg.h":
    ctypedef struct va_list:
        pass
    ctypedef struct fake_type:
        pass
    void va_start(va_list, void* arg)
    void* va_arg(va_list, fake_type)
    void va_end(va_list)
    fake_type int_type "int"

cdef int foo(int n, ...):
    print "starting"
    cdef va_list args
    va_start(args, <void*>n)
    while n != 0:
        print n
        n = <int>va_arg(args, int_type)
    va_end(args)
    print "done"

def call_foo():
    foo(1, 2, 3, 0)
    foo(1, 2, 0)

How do I use a Cython class in a C++ framework

See, for example, http://bitbucket.org/binet/cy-cxxfwk/src and http://groups.google.com/group/cython-users/browse_thread/thread/bc007d85b2ccc518 .


Is there any tool to automatically generate Cython definition files from C (.h) or C++ (.hpp) header files ?

See the main article here.


How do I declare an object of type bool?

Well, that depends on whether you want the C99/C++ bool or the Python bool. Previously, Cython always defaulted to the Python bool type, which led to hard-to-debug issues when users unsuspectingly used bool in wrapping C++ code. We decided to make the choice explicit -- you can import whichever you'd like:

  • For the Python type, do from cpython cimport bool.
  • For the C++ type, do from libcpp cimport bool.

Note that there is also a type called bint, which is essentially a C int but automatically coerces from and to a Python bool value, i.e. cdef object x = <bint>some_c_integer_value gives either True or False.


How can I make a standalone binary from a Python program using cython?

You probably want a recipe something like this:

PYVERSION=2.7
foobar: foobar.py
    cython --embed foobar.py -o foobar.c
    $(CC) -I /usr/include/python$(PYVERSION) foobar.c -lpython$(PYVERSION) -o foobar

The magic is the --embed option, which embeds a copy of the Python interpreter main in the generated C. You'll want to change 'foobar' to reflect the name of your script, of course, and PYVERSION as appropriate.


How to compile Cython with subpackages

Currently, you can't. In order for imports and cimports to just work, you have to compile all your Cython modules from the top-level setup.py file.

This is in part due to the fact that fully qualified names are resolved at compile time, and moving .so files around and adding __init__ files between the Cython compile and the Python runtime invocation means that cimports and imports may resolve differently. Failure to do this may result in errors like .pxd files not found or 'module' object has no attribute '__pyx_capi__'.


Does anyone have a good FAQ format to suggest for this page?


System Message: ERROR/3 (<string>, line 949)

At least one body element must separate transitions; adjacent transitions are not allowed.

CategoryCythonDoc

Docutils System Messages

System Message: ERROR/3 (<string>); backlinks: 1, 2

Anonymous hyperlink mismatch: 2 references but 0 targets. See "backrefs" attribute for IDs.
Something went wrong with that request. Please try again.