Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build project using scikit-build #113

Closed
wants to merge 13 commits into from

Conversation

astrojuanlu
Copy link

This attempts to use scikit-build to build pygmo in an effort to simplify gh-110. For that, I copied tools/wheel_setup.py to setup.py and made some minimal modifications.

I got surprisingly far with comparatively low effort, although the PR is not finished yet:

  • I am copy-pasting the pygmo version into setup.py. With the old process CMake takes care of configuring that - I'm wondering if we could do the same using scikit-build.
  • More importantly, at the moment I'm hardcoding the PYGMO_INSTALL_PATH so that pip install . works on the environment I was using, but using this absolute path is brittle. I am investigating whether a more generic solution can be implemented.

I tested this in a conda environment with the released version of pagmo, which means that I had to temporarily revert 97770b2. However, I am confident it will work with development versions of pagmo as well.

@astrojuanlu
Copy link
Author

astrojuanlu commented Dec 7, 2022

Well, that was surprisingly easy! I modified both CMakeLists.txt files to not use PYGMO_INSTALL_PATH and use a relative directory instead. Arguably some functionality got lost, but now at least both pip install . and python -m build work 🎉

What's left is

  • Confirm that this works with a development version of pagmo2
  • Cleaning up some unused files
  • Adapt the CI
  • Update the installation documentation
  • Rebase WIP: Build and upload wheels on CI #110 on top of this one to verify that the approach indeed works

@astrojuanlu astrojuanlu marked this pull request as ready for review December 7, 2022 14:26
@astrojuanlu astrojuanlu changed the title WIP: Build project using scikit-build Build project using scikit-build Dec 7, 2022
@astrojuanlu
Copy link
Author

@bluescarni @darioizzo this is ready for review.

@astrojuanlu
Copy link
Author

The CI failures are caused by an import error because of the current working directory. Fixing.

@astrojuanlu
Copy link
Author

astrojuanlu commented Dec 7, 2022

There are 5 remaining test failures:

  • 3x NotImplementedError: pool objects cannot be passed between processes or pickled
  • 2x ModuleNotFoundError: No module named '__builtin__'

These are not present in master, so clearly are due to this PR, but it's still not entirely clear to me what might be causing them. These are also present in master at the moment, see gh-114.

@kirbyherm
Copy link
Contributor

just FYI, I am getting the same errors in the build on PR #112 despite it building and running the tests fine on my machine. I'm also in the dark as to what is causing the issue

@astrojuanlu astrojuanlu mentioned this pull request Dec 9, 2022
@bluescarni
Copy link
Member

@astrojuanlu Thanks for taking the time to look into this.

As a general comment, the scikit approach should be an alternative to the CMake build sytem and not a replacement for it. Thus, I would ask if you could place the scikit instructions in a new paragraph in the docs, rather than modifying the existing CMake instructions.

Similarly, the existing conda based builds in the CI pipeline should remain unaltered and new builds testing the scikit approach should be added instead (preferably on github actions, as I think we may be hitting the limits for open source projects on other CI providers).

I'll leave a couple more comments on the diff.

@@ -140,8 +140,6 @@ endif()
find_package(Python3 REQUIRED COMPONENTS Interpreter Development)
message(STATUS "Python3 interpreter: ${Python3_EXECUTABLE}")
message(STATUS "Python3 installation directory: ${Python3_SITEARCH}")
set(PYGMO_INSTALL_PATH "" CACHE STRING "pygmo module installation path")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a useful (albeit niche) option. It would be better if we could keep it. What problems does it cause with scikit?

@@ -1,20 +1,6 @@
# Configure the version file.
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/_version.py.in" "${CMAKE_CURRENT_BINARY_DIR}/_version.py" @ONLY)

# Configure the files needed to make the python wheels (for PyPi packages)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok for removing this.

requires = [
"setuptools>=42",
"scikit-build",
"pybind11",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the rationale for adding pybind11 here? Shouldn't we expect pybind11 to be installed with the other of C++ dependencies?

from setuptools.dist import Distribution
import sys

NAME = 'pygmo'
VERSION = '@pygmo_VERSION@'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not ideal to have to remember to update the version number of pygmo in several different places when making new releases. I'd propose to have a simple txt file in the root directory of the project called pygmo_version containing just 2.18.0 (say) and then reading the version number from this file from both CMake and setup.py.

@darioizzo
Copy link
Member

closing this #117 substitutes it up to when we will consider again scikit-build-core

@darioizzo darioizzo closed this Jan 18, 2023
@astrojuanlu astrojuanlu deleted the scikit-build branch January 18, 2023 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants