Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build a Python package for the client API #71

Merged
merged 13 commits into from Sep 20, 2022

Conversation

varunsh-xilinx
Copy link
Member

Summary of Changes

  • Support building Python wheels with cibuildwheel

Closes #70

Motivation

Using cibuildwheel enables us to build wheels for all supported platforms: currently Python 3.6 - 3.11 on manylinux are supported. Building a native wheel enables users to use Python directly or a package manager like conda as desired.

Implementation

There are two parts to this PR: templatizing the Dockerfile and then supporting cibuildwheel.

To allow the manylinux Docker image to build the server, we need to install our dependencies into it. The image in a CentOS 7 image which was incompatible with our Ubuntu dockerfile. So, the first change is the addition of a generate script that can create a dockerfile that is compatible with Ubuntu (by default, the old behavior is maintained) but now can also generate one that works with a CentOS base image.

Now, that we have a manylinux image that can build our project, the next set of changes are to build the project. There were a few gotchas:

  1. Building in the manylinux image sets _GLIBCXX_USE_CXX11_ABI=0 during compilation. There was one location in the code where that had to be checked. In addition, opentelemetry doesn't seem to work well with this so tracing is entirely removed from the Python wheels until that can be checked further.
  2. The shared library built in manylinux doesn't link to Python so we need to allow undefined symbols when building it. This also requires CMake 3.18+ to check for Development.Module in findPackage(Python...)
  3. For the Python build, don't install any other libraries besides the Pybind11 module. The inclusion of other libraries conflicts with auditwheel during the wheel repair stage for building wheels. The Pybind11 module install location is also changed

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Refactor dockerfile generator

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Update documentation

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
@varunsh-xilinx varunsh-xilinx linked an issue Sep 19, 2022 that may be closed by this pull request
@varunsh-xilinx varunsh-xilinx changed the title 70 build python package for client api Build a Python package for the client API Sep 19, 2022
@gbuildx
Copy link
Collaborator

gbuildx commented Sep 19, 2022

Build Failed! :(

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
@gbuildx
Copy link
Collaborator

gbuildx commented Sep 19, 2022

Build Failed! :(

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
@gbuildx
Copy link
Collaborator

gbuildx commented Sep 19, 2022

Build Failed! :(

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
@gbuildx
Copy link
Collaborator

gbuildx commented Sep 20, 2022

Build Failed! :(

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
@gbuildx
Copy link
Collaborator

gbuildx commented Sep 20, 2022

Build Failed! :(

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
@gbuildx
Copy link
Collaborator

gbuildx commented Sep 20, 2022

Build Failed! :(

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
@gbuildx
Copy link
Collaborator

gbuildx commented Sep 20, 2022

Build Failed! :(

@varunsh-xilinx
Copy link
Member Author

The build passed after some debugging internally

@varunsh-xilinx varunsh-xilinx merged commit 134cd03 into main Sep 20, 2022
@varunsh-xilinx varunsh-xilinx deleted the 70-build-python-package-for-client-api branch September 20, 2022 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build Python package for client API
2 participants