enhancements cpp

Improved support for C++

This was being worked on as part of a Google Summer of Code project by Danilo Freitas mentored by Robert Bradshaw, and will be released as part of Cython 0.13. See gsoc09/daniloaf and gsoc09/daniloaf/progress.

<<TableOfContents(2)>>

Heap allocation

Status

The syntax is:

cdef A* a = new A(...)
del a

Discussion

(This is supported through ugly hacks only)

There's no support for the new and delete operators. Possible syntax:

# first declare A as an extern C++ class

# Option 1
cdef A* a
a = A(constructor args) # should this be used for stack alloc instead?
                        # see next section
...
del A # would we use deleting a pointer variable for anything else?

# Option 2
cimport cython
cdef A* a
a = cython.new(A, constructor args)
...
cython.delete(a)

# Others?

Field allocation

Currently using a C++ class as a member field of a struct or Cython class is probably not a good idea; whether or not the constructor or destructor will be called probably depends on the situation (a struct allocated on the stack in a function could work OK if there's a constructor with no arguments of the C++ class field; for malloc-ed structs or Cython classes the constructor/destructor won't be called).

On may have to do some trickery with overriding the new operator etc. in order to support this through Cython class constructors. But it would be preferably to emit C++ constructors and use new/delete all the way.

Stack allocation

The only thing that is working is calling a constructor without arguments upon function entry and the destructor at function exit. This is not sufficient; e.g. it prevents keeping STL iterators on the stack.

There's a fundamental mismatch here between Python scoping and C++ scoping. C++ constructors and destructors are actively called as part of C++ scoping:

if (x) {
   DatabaseConnection conn("localhost", 8080); // constructor called
   conn.query(...)
   ...
   // destructor called
} else {
   ...
}

Introducing these rules in Cython would be incredibly confusing, as it doesn't match Python scoping rules.

So here's an alternative proposal:

First, disable stack allocation altogether for C++ classes; heap allocation should be used.

Being a lot more ambitious, this would be very nice:

1. A variable typed as a C++ class (like cdef MyCppClass myvar) has the same semantics as Cython classes, i.e. a reference-counted reference. It does not mean the same as when a struct is declared (so one must seperate strongly between C++ classes and structs).
2. The implementation is thus probably either a small wrapper to make it a Python object (but with direct calls to the C class!), or a custom reference count scheme with less memory overhead. (Note that with a dependency on Boost++, one can just use a smart_ptr. Note that a smart_ptr comes with C++0x.)
3. At a later point, for optimization, one can then use live variable analysis to identify blocks where stack allocation using a C++ block is possible.

Example:

cdef extern "C++":
    class MyCppClass: # MUST have syntax different from struct
        MyCppClass(int n) # Constructor
        void hello()

cdef MyCppClass f(x):
    cdef MyCppClass a, b, c
    if x:
        a = MyCppClass(1) # Used outside of conditional assignment, so refcount
        b = MyCppClass(2) # This starts a new block " { MyCppClass __pyx_v_b(2); ..."
        c = MyCppClass(3)
        b.hello()
        b = None
        # End of block for b, as b's destructor would have been called in Python/Cython
        if x.y: c = None # This is not expressible with stack allocation, so refcount
    print c is None # Can be None
    return a # Just works because of refcounting

Upcasts

Cython doesn't know about the type hierarchy, so upcasts won't work automatically. One should be able to do:

cdef extern ...:
    # somehow declare C++ classes A and B, and that B inherits from A

cdef foo(A* a): ...

cdef B* b = ...
foo(b) # should just work

Note that C++ has multiple inheritance, which is trivial to support here.

Const

This is shared with C; but there's a more solid tradition for using const in C++.

Casts

C++ casting is not supported. E.g.

cimport cython
a = cython.cpp.reinterpret_cast(A, b)
a = cython.cpp.const_cast(A, const_a)
a = cython.cpp.static_cast(A, b)
a = cython.cpp.dynamic_cast(A, b)

References

Cython knows nothing of C++ references. This gives some odd results; consider

C++:
  void incargs(int& a, int& b) { a++; b++ }
Cython:
  cdef extern void incarg(int a, int b)

  cdef int a = 1
  cdef object b = 1
  incargs(a, b)
  # Now, a is 2 but b is 1!!

Suggested solution: Having a simple f(x) modify x has a very unpythonic feel and should probably be avoided. So the suggestion is to allow the reference declaration in Cython, but then treat them as pointers! The same C++ function would the be called like this:

Cython:
  cdef extern void incarg(int& a, int& b)

  cdef int a = 1, b = 1
  incarg(&a, &b)
  # It is now impossible to pass a temporary as a reference, so
  # one will automatically not attempt to pass Python objects

Generated C:
   incarg(*(&(a)), *(&(b)))

So Cython demands a pointer and then dereference it, which again gets its address taken by the C++ compiler :-)

This applies to return values too. In C++ functions can return mutable references, so that i.e. f() = 4 makes sense if f has return type int&. Such functions would be treated as returning a pointer in Cython which allows dereffing it with [0]. If such a pointer is assigned to another variable we must generate code to take the addresss.

cdef extern from "foo.h" namespace "Project::Foo":
    ...

Templates

...

Overloaded operators

Status

Implemented as below, with C++ style operatorX declarations.

Discussion

(Note: This was written spring 2008 and the use of & below contradicts the solution for references scetched above)

It would be convenient if one could declare that operators are overloaded on a C++ class so that Cython rather than give a compiler error will let the operator use through to C++.

One should probably (?) keep this apart from any operator overloading as such (i.e. within Cython), because overriding operators in Cython will mean different code is generated by Cython while declaring that operators are overloaded in C++ is part of the description on what one is dealing with C++-side. Trying to merge the two seem to lead to lots of problems with cooking up a syntax that disappears when one treat them as seperate things.

Using the C++ syntax directly gives exactly the right expression power, however it does require some changes to the parser.

cdef extern from "Rectangle.h":
    ctypedef struct Rectangle:
        Rectangle(int) # constructor takes an int
        Rectangle operator+(Rectangle)
        Coordinate& operator[](Corner corner) # & means result is assignable

Exceptions

Currently, we can translate (external) C++ exceptions into Python exceptions with the except + syntax. It may be nice to be able to raise and catch these things in Cython directly (perhaps using try...except). Also, it could be good to provide proper cleanup when such exceptions are propagated (currently it's possible to leak references).