Evaluate the pros/cons of using ctypes to replace pybind11

[k-ye]

While pybind11 is quite easy to use, it comes with a few pitfalls:

• Large Python overhead
• Debug message is not very intuitive

Many large projects have switched to use ctypes. Writing this down to remind us that we should evaluate its feasibility.

[bobcao3]

One plus side is PyPy supports ctypes well, another is that combined with AOT API now we can also expose device API as well

[yuanming-hu]

Two more pros:

• Having a C API for ctypes may help us build other frontends, such as rust?
• Using ctypes allows a single shared object to be used for all Python versions. (Not sure how to benefit from it though.)

Debug message is not very intuitive

That's true, but I don't really think this is a con for users, since Taichi should not throw FFI errors to users. Probably for developers only :-)

One cent about switching to ctypes: maybe we can progressive replace the interface, and replace the parts about kernel launches first. That will significantly lower the kernel invocation overhead. For the AST construction part, we can replace them at a lower priority.

[masahi]

• Large Python overhead

The author of pybind is working on a new python binding project, https://github.com/wjakob/nanobind, one of the motivations being reducing overhead.

[ailzhang]

FYI I wrote a few toy demos of calling c extensions from python using pybind11/ctypes/cpython respectively and the code lives at https://github.com/ailzhang/c_ext_for_py/. It also prints time spent (maybe not apple-to-apple comparison tho).
I've noticed a few things while trying this out:

1. Depending on the real workload, ctypes might have up to 2x speedup compared to pybind11 and cpython is usually the fastest among the three (with some exceptions probably due to how I wrote the demos :P
2. The speedup can be easily ruined if some extra work is done in python sigh. For all three approaches, the more you do in python, the slower it is.

As a result I'd view this issue more to identify hotspots in py->c interaction and migrate them to cpython/ctypes step by step in a measurable way. We can probably employ cpython/ctypes in the critical parts for perf gain and keep some components in pybind11 to enjoy the ready-to-use C++ features.

[bobcao3]

We should move AOT first to CTypes, and then expose device API over ctypes.