Provide Python bindings for `overlap` #13

Helveg · 2020-12-31T15:54:20Z

Hi there! Your overlap library has many more applications than you might imagine :) I'd like to use it in the data science world which is Python dominated. Is it possible to provide a compiled version of this as a Python package? There are things like pybind that will automatically generate Python bindings from your header file.

I have 0 experience on the C++ side but would be interested to help you out wherever I can on the Python side, testing, packaging etc!

The text was updated successfully, but these errors were encountered:

severinstrobl · 2020-12-31T16:30:02Z

Interestingly enough, that is actually something I thought about recently, but didn't really have a use-case for so far. Generating Python wrappers using pybind11 should be pretty straight forward based on some past experiments. Now that I know there are potential users for such bindings, I'll look into this asap. I'll keep you posted and would certainly appreciate help in testing, etc.!

severinstrobl · 2021-01-02T13:13:21Z

Just as a heads up, I got the basic Python bindings working using pybind11. I'm currently sorting out some tests and packaging, a first version to experiment with should be available soon.

Helveg · 2021-01-02T14:08:51Z

Supercool! :) I can experiment with it and write the docs for it as I go along!

severinstrobl · 2021-01-03T23:05:05Z

A first version of the Python bindings are available in the branch python, if you have the chance to give it a try I'd appreciate any feedback.

Helveg · 2021-01-04T10:08:46Z

It seems my Windows Subsystem for Linux can't build the wheel because Eigen3 is missing.

Here's the full error of pip install .:

robin@LAPTOP-N6QKV63M:~/git/overlap$ pip install .
Processing /mnt/c/Users/robin/Documents/GIT/overlap
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: numpy in /home/robin/.pyenv/versions/3.8.6/lib/python3.8/site-packages (from overlap==0.0.5) (1.19.4)
Building wheels for collected packages: overlap
  Building wheel for overlap (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/robin/.pyenv/versions/3.8.6/bin/python3.8 /home/robin/.pyenv/versions/3.8.6/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpl9o_0r9u
       cwd: /tmp/pip-req-build-xx77w_1x
  Complete output (110 lines):
  Not searching for unused variables given on the command line.
  CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-req-build-xx77w_1x/_cmake_test_compile/build/CMakeFiles/CMakeOutput.log".
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 9.3.0
  -- Check for working C compiler: /usr/bin/cc
  -- Check for working C compiler: /usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is GNU 9.3.0
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-req-build-xx77w_1x/_cmake_test_compile/build
  -- The CXX compiler identification is GNU 9.3.0
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  CMake Error at CMakeLists.txt:15 (find_package):
    By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has
    asked CMake to find a package configuration file provided by "Eigen3", but
    CMake did not find one.

    Could not find a package configuration file provided by "Eigen3" (requested
    version 3.0.0) with any of the following names:

      Eigen3Config.cmake
      eigen3-config.cmake

    Add the installation prefix of "Eigen3" to CMAKE_PREFIX_PATH or set
    "Eigen3_DIR" to a directory containing one of the above files.  If "Eigen3"
    provides a separate development package or SDK, be sure it has been
    installed.


  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-req-build-xx77w_1x/_skbuild/linux-x86_64-3.8/cmake-build/CMakeFiles/CMakeOutput.log".
    File "/tmp/pip-build-env-akck6dlk/overlay/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 582, in setup
      env = cmkr.configure(cmake_args,
    File "/tmp/pip-build-env-akck6dlk/overlay/lib/python3.8/site-packages/skbuild/cmaker.py", line 229, in configure
      raise SKBuildError(


  --------------------------------------------------------------------------------
  -- Trying "Ninja" generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying "Ninja" generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying "Unix Makefiles" generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying "Unix Makefiles" generator - success
  --------------------------------------------------------------------------------

  Configuring Project
    Working directory:
      /tmp/pip-req-build-xx77w_1x/_skbuild/linux-x86_64-3.8/cmake-build
    Command:
      cmake /tmp/pip-req-build-xx77w_1x -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-req-build-xx77w_1x/_skbuild/linux-x86_64-3.8/cmake-install/python -DPYTHON_EXECUTABLE:FILEPATH=/home/robin/.pyenv/versions/3.8.6/bin/python3.8 -DPYTHON_VERSION_STRING:STRING=3.8.6 -DPYTHON_INCLUDE_DIR:PATH=/home/robin/.pyenv/versions/3.8.6/include/python3.8 -DPYTHON_LIBRARY:FILEPATH=/home/robin/.pyenv/versions/3.8.6/lib/libpython3.8.so -DSKBUILD:BOOL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-akck6dlk/overlay/lib/python3.8/site-packages/skbuild/resources/cmake -DCMAKE_BUILD_TYPE:STRING=Release

  Traceback (most recent call last):

  An error occurred while configuring with CMake.
    Command:
      cmake /tmp/pip-req-build-xx77w_1x -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-req-build-xx77w_1x/_skbuild/linux-x86_64-3.8/cmake-install/python -DPYTHON_EXECUTABLE:FILEPATH=/home/robin/.pyenv/versions/3.8.6/bin/python3.8 -DPYTHON_VERSION_STRING:STRING=3.8.6 -DPYTHON_INCLUDE_DIR:PATH=/home/robin/.pyenv/versions/3.8.6/include/python3.8 -DPYTHON_LIBRARY:FILEPATH=/home/robin/.pyenv/versions/3.8.6/lib/libpython3.8.so -DSKBUILD:BOOL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-akck6dlk/overlay/lib/python3.8/site-packages/skbuild/resources/cmake -DCMAKE_BUILD_TYPE:STRING=Release
    Source directory:
      /tmp/pip-req-build-xx77w_1x
    Working directory:
      /tmp/pip-req-build-xx77w_1x/_skbuild/linux-x86_64-3.8/cmake-build
  Please see CMake's output for more information.
  ----------------------------------------
  ERROR: Failed building wheel for overlap
Failed to build overlap
ERROR: Could not build wheels for overlap which use PEP 517 and cannot be installed directly

Helveg · 2021-01-04T10:09:33Z

You could use cibuildwheel to build and distribute wheels during your CI pipelines.

severinstrobl · 2021-01-04T16:46:58Z

Sorry, I didn't realize you're on Windows where getting development packages installed might be a bit trickier. One option would be to include eigen3 as a third-party module. I didn't do that so far as eigen3 is a common component in many numerical frameworks and having two instances of eigen3 of potentially different versions around is just asking for trouble.
But for the Python bindings this should not be an issue, so at least in this case including a copy of eigen3 would be a viable option.
Thanks for the idea with cibuildwheel, that seems just like what I was looking for! I'll try to get that working next.

Helveg · 2021-01-04T17:12:28Z

I think that it is common now for python packages to be provided as wheels exactly because it is so difficult to build software on every machine where Python is available 😛 I hope the cibuildwheel option can help you transform setup.py into wheels smoothly! If you run into issues there I can help, if you're low on time it is something I could probably do in a PR as well!

severinstrobl · 2021-01-04T21:39:57Z

Looks like I managed to sort most of the issues out, eigen3 is now included as a sub-module which simplifies things. The wheels seem to be building now for all platforms 🤞

severinstrobl · 2021-01-08T18:46:15Z

So I finally managed to sort the Python wheels out, sorry it took way longer than I had hoped. I uploaded the package to TestPyPI, maybe you can give it a go? There should be wheels for the 3 major OSes and the typically used versions of Python. Once I'm somewhat confident everything works as intended I'll push it to the official PyPI.

Helveg · 2021-01-09T20:22:06Z

Hah don't worry, you're still way ahead on the curve of providing wheels! Massive thanks for the massive effort :)

Looking in indexes: https://test.pypi.org/simple/
Collecting overlap
  Downloading https://test-files.pythonhosted.org/packages/4c/07/282b2529e9ea915b7f1f534d5562898a7b0bd1252d7d70cae36780a8e351/overlap-0.0.5-cp39-cp39-win_amd64.whl (130 kB)
     |████████████████████████████████| 130 kB 504 kB/s
Requirement already satisfied: numpy in c:\users\robin\appdata\local\programs\python\python39\lib\site-packages (from overlap) (1.19.4)
Installing collected packages: overlap
Successfully installed overlap-0.0.5
WARNING: You are using pip version 20.2.4; however, version 20.3.3 is available.
You should consider upgrading via the 'c:\users\robin\appdata\local\programs\python\python39\python.exe -m pip install --upgrade pip' command.
PS C:\Users\robin> python
Python 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import overlap
>>> dir(overlap)
['Hexahedron', 'Sphere', 'Tetrahedron', 'Wedge', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_overlap', 'overlap', 'overlap_area']

The install works like a charm :)

severinstrobl · 2021-01-09T20:26:24Z

Until there is some real documentation, you can have a look at the Python tests.

Helveg · 2021-02-09T11:35:14Z

Right, when I have the bandwidth for it I'll be adding this into https://pypi.org/project/radialdf/!

severinstrobl changed the title ~~Using overlap in data science (i.e. Python)~~ Provide Python bindings for overlap Jan 2, 2021

severinstrobl added this to the 0.1.0 milestone Jan 2, 2021

severinstrobl added the enhancement label Jan 2, 2021

severinstrobl mentioned this issue Feb 7, 2021

Python bindings #19

Merged

severinstrobl closed this as completed in #19 Feb 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide Python bindings for `overlap` #13

Provide Python bindings for `overlap` #13

Helveg commented Dec 31, 2020 •

edited

severinstrobl commented Dec 31, 2020

severinstrobl commented Jan 2, 2021

Helveg commented Jan 2, 2021

severinstrobl commented Jan 3, 2021

Helveg commented Jan 4, 2021 •

edited

Helveg commented Jan 4, 2021

severinstrobl commented Jan 4, 2021

Helveg commented Jan 4, 2021

severinstrobl commented Jan 4, 2021

severinstrobl commented Jan 8, 2021

Helveg commented Jan 9, 2021

severinstrobl commented Jan 9, 2021

Helveg commented Feb 9, 2021

Provide Python bindings for overlap #13

Provide Python bindings for overlap #13

Comments

Helveg commented Dec 31, 2020 • edited

severinstrobl commented Dec 31, 2020

severinstrobl commented Jan 2, 2021

Helveg commented Jan 2, 2021

severinstrobl commented Jan 3, 2021

Helveg commented Jan 4, 2021 • edited

Helveg commented Jan 4, 2021

severinstrobl commented Jan 4, 2021

Helveg commented Jan 4, 2021

severinstrobl commented Jan 4, 2021

severinstrobl commented Jan 8, 2021

Helveg commented Jan 9, 2021

severinstrobl commented Jan 9, 2021

Helveg commented Feb 9, 2021

Provide Python bindings for `overlap` #13

Provide Python bindings for `overlap` #13

Helveg commented Dec 31, 2020 •

edited

Helveg commented Jan 4, 2021 •

edited