Is there a Python binding to libpointmatcher

[s0r2637]

I was wondering if there is a Python binding to libpointmatcher?

[pomerlef]

Clearly not, but I would highly appreciate it ;)

[pomerlef]

We starting to work on Python binding. If someone has some inputs, advices or needs. Now is the time to tell us!

[PhiBabin]

There was the start of an implementation of python bindings. It supported registration, add numpy array as pointcloud input and support extracting the debugging information that are normally in the .vtk output. However, it needs to be clean up, there is a bunch of code that was made specifically for debugging penalized ICP. https://github.com/norlab-ulaval/test_penalized_icp

[YoshuaNava]

@PhiBabin How did you attempt the Python binding in your branch?

@pomerlef Two questions:

• Do you plan to support all interpreters of Python?
• Will the full API of libpointmatcher be covered?

I previously worked on making the Python bindings of Kalibr build and ship as released software (Debians). They included marshalling for eigen/numpy datatypes, wrappers around classes and function calling directly from Python. Making them work, after managing all the build targets was straightforward.

---

Overview of tools

This is a list of solutions taken by different libraries:

• gtsam - Cython.
• librealsense and Open3D - pybind11
• Kalibr - boost.python.

Things to take into account:

• Compile time pybind11 vs boost.python - https://pybind11.readthedocs.io/en/stable/limitations.html
• Both pybind11 and boost.python might be slower than cython [pybind issue]

[aguenette]

@YoshuaNava Thanks for the informations! I'll take a look at them! 🙂

@PhiBabin I don't have access to the repo you've mentionned. 🙁

[PhiBabin]

@YoshuaNava I used boost.python. I only supported the most basic usecase of registering two pointclouds. Only a small subset of the api was implemented.
@AlexandreG87 Oups, my bad, it is now public.

[aguenette]

Alright! So, after researching about all the Python bindings tools available, here are the ones I consider to have good potential to do the job. These solutions have been selected base on the following criteria :

Primary criteria:

• Performance
• Modern C++ and Python3 compatibility
• Effective Numpy to Eigen conversion support (not 100% sure for cppyy)
• Template compatibility
• The ease of use and maintenance
• The solution must still be actively maintained
• No or minimal modification to the original code must be done

Secondary criteria:

• Availability/Quality of documentation and tutorials
• Control and flexibility over interface implementation
• Does not require too much dependency

The fully automatic solutions :

• Automatic Wrapper and Interface Generator (AutoWIG) :

  • Article on AutoWIG
  • YouTube video explaning how AutoWIG works
  • GitHub repo

• cppyy :

  • Paper on cppyy
  • Bitbucket repo
  • Tutorial
  • Stack Overflow answer by the author

And the semi-automatic solutions :

• Pybind11 :

  • GitHub repo
  • JaggedVerge tutorial
  • Real Python tutorial
  • YouTube playlist tutorial
  • Integrate Python and C++ with pybind11 - Robert Smallshire

• Boost.Python :

  • GitHub repo
  • Official tutorial
  • Developpez.com tutorial (French tutorial)
  • Integrating Python and C++ with Boost Python part 1 | SciPy 2014 | Austin Bingham
  • Integrating Python and C++ with Boost Python part 2 | SciPy 2014 | Austin Bingham
  • Integrating Python and C++ with Boost Python part 3 | SciPy 2014 | Austin Bingham
  • Integrating Python and C++ with Boost Python part 4 | SciPy 2014 | Austin Bingham

• Cython

  • Cython docs :
  • Cython GitHub repo
  • Cython Wiki
  • Real Python tutorial
  • Paper (IEEE Xplore access required)

List of projects and examples using these tools :

• AutoWIG :

  • StatisKit
  • FP17

• cppyy :

  • Example repos from the docs

• Pybind11 :

  • Tensorflow (Use Eigen too)
  • Pytorch (Use Eigen too)
  • NumpyEigen
  • Open3D (Use Eigen too)
  • CMake examples
  • Python examples
  • Other examples
  • pclpy

• Boost.Python :

  • EigenPy
  • MiniEigen
  • Examples

• Cython :

  • Eigency
  • SageMath
  • Numpy
  • python-pcl

Some more stuff on the subject :

• Real Python bindings tutorial
• Blog - Which tool should you use? by Stephan Behnel (Cython core developper) (Not just about Cython!)
• Examples for different Python bindings tools
• Calling C/C++ from Python? - Stack Overflow

In short...

Well, all of the above tools have their strenghs and weaknesses, none of them are perfect. But, I think it's worth trying the fully automatic solutions first, since they are designed with the ease of use and maintenance in mind. On the other hand, AutoWIG generate Pybind11 or Boost.Python bindings, both of whom have extensive of tutorials, documentations and examples available. So, in the event AutoWIG is harder to grasp than Pybind11 or Boost.Python, then I can go with either of these tools. All of these tools are focused on performance, so I'm not worried about that too much. The main struggle being the Python call overhead, except for the PyPy interpreter. cppyy, Pybind11 and Cython are compatible with the PyPy interpreter, although cppy seems to get a bit more out of it than the other ones.

Finally

Every suggestions, comments or advices are welcome. Feel free to suggest anything that could make this project even better, the goal is to be able to provide good bindings for libpointmatcher. Thanks!

[pomerlef]

Thanks for the thorough analysis. I guess we can start with AutoWIG.

@simonpierredeschenes any concerns?

[wlav]

Just adding a couple of remarks; feel free to ignore. :) Full disclosure: I'm the author of cppyy, and saw some of my documents referenced above, which is how I landed here.

First: "all of the above tools have their strenghs and weaknesses". This is an extremely important point as you have "Performance" on the top of your list. Performance in CPython (and PyPy even more so), from an automatic binder, depends on specializing the paths that you care about. If that is already happening out-of-the-box, then you're golden. If not, you're in for some work. I'm afraid that testing and profiling is the only way to find out.

On that point, yes, automatic binders do help a lot with maintenance, but even though they get you 99% of the way, that last bit that they lack is also the hardest part. I.e. compared to a full manual solution, that 1% would be more like 20-30% of the manual effort. Still a gain for sure, but you may be forced to learn more about the tool than you hoped for.

For Eigen/NumPy: note that Eigen's layout may differ depending on compiler settings (e.g. if you enable vector instructions), so simply copies may be the only portable way in some cases. As for cppyy, one problem with Eigen is that template expressions are intended to be squashed by the C++ compiler. This does not naturally happen in cppyy in the way that wrappers are generated, so sometimes you end up with massive expressions escaping to Python. I'm working on Eigen-specific module, but this is a WIP.

Finally, for PyPy, cppyy there has less functionality than on CPython. It will trivially outperform the other solutions, but it does so because there is builtin support for cppyy in PyPy. The others have to go through the emulated C-API which can be very slow. This builtin cppyy module is being separately maintained and because of the higher demand for CPython support, effort has mainly been going to CPython, so cppyy in PyPy is now lagging.

[pomerlef]

Thanks a lot @wlav ! Your inputs are much appreciated. We will continue our analysis, but I guess we need to do some quick tests with those tools to understand a bit more what is going on.

[aguenette]

Yes! Thank you very much @wlav! Your feedback is very much appreciated!