Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
408 lines (340 sloc) 11.6 KB

Use Python Objects in C++

Date: 2018/5/6

C++ is difficult. The language has so many catches and requires us to take care of every detail. The reason to use it is usually performance. But learning C++ is so frustrating that when performance isn't mandatory, people try hard to avoid it. Things are getting better now. Recently, C++11 standard looks just like the right way to go for the language. C++ is still hard, but the learning path isn't as concealed as it did.

It now makes sense to encourage a Python programmer to invest in C++. These are two very different languages. Each is good in its field. Although both are pronounced and designed to be general-purpose, and indeed are used for everything, it's naive to use vanilla Python for anything calling for speed. On the other hand, C++ is too heavy-lifting for simple one-liners or scripts. To master the two very different languages gives a programmer powerful synergy. There are alreay many wrappers developed to bridge them. Python the interpreter is also a well-organized C library easy to be used from C++. The major thing that Python programmers concerned was the complexity of C++ the language. With C++11, it is very much mitigated.

And we also have pybind11, a compact library providing comprehensive wrapping between Python and C++11. It also includes neat API for manipulating Python objects. The API is very basic and covers only a fraction of what Python C API does, but fun to use. And it's the real point of this post: make a note about manipulating Python objects with pybind11. Other parts of the library are important and probably more useful than this. But I find this particularly interesting.

I am starting with "hello, world". But it's boring to just print a Python :py:class:`str`, which doesn't differ much from C++ :cpp:class:`std::string`. Let me use a container to do it:

That's how you create a Python :py:class:`list` in C++. And you see how it's returned to Python and gets the famous hello, world:

:cpp:class:`pybind11::list`

More on :cpp:class:`pybind11::list`. Take a look at https://github.com/pybind/pybind11/blob/master/include/pybind11/pytypes.h, it's just a thin shell to access :c:type:`PyList` API. The pybind11 support isn't fancy, but enough for basic operations.

Run the code. Elements are converted to :cpp:class:`std::string` and sent to standard output one by one:

pybind11 provides :cpp:class:`pybind11::list::append` to populate elements (we saw it in the hello, world). Spell it out:

This is the result:

:cpp:class:`pybind11::tuple`

:py:class:`tuple` is immutable and more restrictive than :py:class:`list`. pybind11 provides API for reading it. To creat a non-trivial tupple, we can convert from a sequence object:

Execution in Python:

:cpp:class:`pybind11::dict`

:cpp:class:`pybind11::dict` is slightly richer than the sequences. This is how to create a :py:class:`dict` from a :py:class:`tuple` in C++:

Result:

In addition to the obvious :cpp:func:`pybind11::dict::size`, it has :cpp:func:`pybind11::dict::clear` and :cpp:func:`pybind11::dict::contains`. The second example uses them to process the created dict:

Then the dictionary becomes empty:

:cpp:class:`pybind11::str`

I've used :cpp:class:`pybind11::str` many times in previous examples. Here I just bring up one more trick: C++11 literal for strings.

Result:

:cpp:class:`pybind11::handle` and :cpp:class:`pybind11::object`

:cpp:class:`pybind11::handle` is a thin wrapper in C++ to the Python :c:type:`PyObject`. It's the base class of all pybind11 classes that wrap around Python types.

:cpp:class:`pybind11::object` is derived from :cpp:class:`pybind11::handle`, and adds automatic reference counting. The two classes offer bookkeeping for Python objects in pybind11.

See the change of the reference count.

:cpp:class:`pybind11::none`

The last class covered in this note is :cpp:class:`pybind11::none`. It is just the :py:obj:`None` object, or in the C API :c:type:`Py_None`. :py:obj:`None` is also reference counted, and it's convenient that in pybind11 we have a class representing it.

See the test result:

Reference

List of Python types supported in pybind11: https://pybind11.readthedocs.io/en/stable/advanced/pycpp/object.html.