Skip to content

Rewrite python binding CMake configuration and CI checks#483

Merged
cary-ilm merged 7 commits intoAcademySoftwareFoundation:mainfrom
cary-ilm:python-cmake-ci-rewrite-rebase
May 13, 2025
Merged

Rewrite python binding CMake configuration and CI checks#483
cary-ilm merged 7 commits intoAcademySoftwareFoundation:mainfrom
cary-ilm:python-cmake-ci-rewrite-rebase

Conversation

@cary-ilm
Copy link
Copy Markdown
Member

@cary-ilm cary-ilm commented Apr 29, 2025

[NOTE: Reminder that the WIP rewrite of the python bindings using pybind11 are in parallel with the existing boost.python bindings, in src/pybind11 and src/python respectively, with the plan to retire the src/python boost-based bindings when the pybind11 implementation is complete. This configuration rewrite applies to both.]

This does four things:

  1. Complete rewrite of the CMake configuration for the python bindings. This drops support for Python2, and it eliminates src/python/config/ModuleDefine.cmake altogether and moves the CMake code directly into PyImath, PyImathNumpy, etc. It's more direct and easier to follow this way.

  2. Add CI checks to build and test the python bindings on macOS and Windows, previously not done. The python bindings require boost.python, which is now installed via conan. A few caveats on Windows:

    a. The Windows Debug build does not currently work. It stubbornly requires the debug library python39_d.lib, and I can't figure out how to redirect it to python39.lib. None of the suggested strategies seem to work, so the build is disabled for now. This is left as a TODO.

    b. The python and pybind11 bindings build successfully on Windows, but they don't currently execute properly, the "import" checks fail. I suspect a link/symbol error, but the message is unhelpful. The PATH and PYTHONPATH appear to be set correctly and as far as I can tell the necessary dlls are in place. This is left as a TODO (several TODO's actually).

  3. Configure PyImath as a COMPONENT of Imath, to address Please consider making the python bindings optional #360 and Alembic Windows compilation Python3.9.10 called with non-existent target "Imath::PyImath_Python3_9". #395. Downstream projects that require PyImath must now specify:

    find_package(Imath COMPONENTS PyImath)

    whereas projects that simply say:

    find_package(Imath)

    will no longer generate a dependency on PyImath. Previously, the only way to avoid this dependency was to build/install Imath with the python bindings disabled. Please consider making the python bindings optional #360 was based on the hope/assumption that optional downstream dependence on the python bindings could be configured at Imath build/install time, but the desired behavior can only be accomplished by introducing a new downstream configuration option.

    The CI validates this by building the ImathTest and PyImathTest projects as standalone applications linked against the just-installed Imath/PyImath build, confirming both that they compile/link and that they have the proper dependencies.

  4. Fixes the Apple Framework iOS cross-compilation option, and adds that to the CI as well.

@cary-ilm cary-ilm force-pushed the python-cmake-ci-rewrite-rebase branch from 8e25911 to 995fd43 Compare May 1, 2025 03:04
@Treata11
Copy link
Copy Markdown
Contributor

@cary-ilm
I tested this branch & the frameworks were generated successfully without a suffix & with proper bundle-IDs.
However, ImathConfig.h wasn't included in the Headers directory which is best to be kept that way (as we discussed it earlier).
Also, the resources (such as ImathTargets, ImathConfig ...) were not bundled in the framework's Resources dir as I handled in #466. It'll be handy building downstream projects for iOS targets.

I'll be glad to submit a fix for these minor issues, but I think it might be best to keep that for a separate PR after this is merged.

@cary-ilm
Copy link
Copy Markdown
Member Author

@Treata11, here's the log of what was installed, from the log of the job that builds the framework:

Install configuration: "Release"
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Imath
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Resources
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Resources/LICENSE.md
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Resources/ImathConfig.cmake
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Resources/ImathTargets.cmake
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Resources/README.md
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Resources/ImathConfigVersion.cmake
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Resources/SECURITY.md
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathMatrixAlgo.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/half.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/halfFunction.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathFrustumTest.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathPlatform.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathBoxAlgo.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathFun.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathRoots.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathInterval.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathVecAlgo.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathForward.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathEuler.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathColorAlgo.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathRandom.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathLineAlgo.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathGLU.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathInt64.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathTypeTraits.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathShear.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathLine.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/halfLimits.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathConfig.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathGL.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathExport.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathVec.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathSphere.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathColor.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathMath.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathPlane.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathFrame.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathBox.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathFrustum.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathMatrix.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathNamespace.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Headers/ImathQuat.h
-- Installing: /Users/runner/work/Imath/Imath/_install/lib/Imath.framework/Info.plist

The .cmake files appear to be under Resources and ImathConfig.h appears to be under Headers. Are you seeing something different, or am I missing something?

@Treata11
Copy link
Copy Markdown
Contributor

@cary-ilm

I only checked the installed artifacts in the build-dir at first & didn't look at /usr/local/lib/...
All should be fine then.

cary-ilm added 7 commits May 12, 2025 18:04
This does three things:

1. Complete rewrite of the CMake configuration for the python
   bindings. This drops support for Python2, and it eliminates
   src/python/config/ModuleDefine.cmake altogether and moves the CMake
   code directly into PyImath, PyImathNumpy, etc. It's more direct and
   easier to follow this way.

2. Add CI checks to build and test the python bindings on macOS and
   Windows, previously not done. The python bindings require
   boost.python, which is now installed via conan. A few caveats on
   Windows:

   a. The Windows Debug build does not currently work. It stubbornly
      requires the debug library python39_d.lib, and I can't figure
      out how to redirect it to python39.lib. None of the suggested
      strategies seem to work, so the build is disabled for now. This
      is left as a TODO.

   b. The python and pybind11 bindings build successfully on Windows,
      but they don't currently execute properly, the "import" checks
      fail. I suspect a link/symbol error, but the message is unhelpful. The
      PATH and PYTHONPATH appear to be set correctly and as far as I
      can tell the necessary dlls are in place. This is left as a TODO
      (several TODO's actually).

3. Configure PyImath as a COMPONENT of Imath, to address AcademySoftwareFoundation#360 and
   AcademySoftwareFoundation#395. Downstream projects that require PyImath must now specify:

      find_package(Imath COMPONENTS PyImath)

   whereas projects that simply say:

      find_package(Imath)

   will no longer generate a dependency on PyImath. Previously, the
   only way to avoid this dependency was to build/install Imath with
   the python bindings disabled.  AcademySoftwareFoundation#360 was based on the hope/assumption
   that optional downstream dependence on the python bindings could be
   configured at Imath build/install time, but the desired behavior
   can only be accomplished by introducing a new downstream
   configuration option.

   The CI validates this by building the ImathTest and PyImathTest
   projects as standalone applications linked against the
   just-installed Imath/PyImath build, confirming both that they
   compile/link and that they have the proper dependencies.

Signed-off-by: Cary Phillips <cary@ilm.com>
Signed-off-by: Cary Phillips <cary@ilm.com>
Signed-off-by: Cary Phillips <cary@ilm.com>
Signed-off-by: Cary Phillips <cary@ilm.com>
Signed-off-by: Cary Phillips <cary@ilm.com>
Signed-off-by: Cary Phillips <cary@ilm.com>
Signed-off-by: Cary Phillips <cary@ilm.com>
@cary-ilm cary-ilm force-pushed the python-cmake-ci-rewrite-rebase branch from c740782 to 14134a8 Compare May 13, 2025 01:50
@cary-ilm
Copy link
Copy Markdown
Member Author

Merging as is, we can follow up later as needed.

@cary-ilm cary-ilm merged commit de8553c into AcademySoftwareFoundation:main May 13, 2025
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants