Skip to content

chore(build): build wheels for multiple platforms#281

Merged
jericht merged 1 commit into
OpenJobDescription:mainlinefrom
jericht:jericht/multi_platform_build
Jun 23, 2026
Merged

chore(build): build wheels for multiple platforms#281
jericht merged 1 commit into
OpenJobDescription:mainlinefrom
jericht:jericht/multi_platform_build

Conversation

@jericht

@jericht jericht commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

What was the problem/requirement? (What/Why)

The package now contains a native Rust extension (PyO3 bindings). Previously, openjd-model was pure Python and a single py3-none-any wheel worked on all platforms. With native code, we need to build and publish one wheel per platform so that pip install works without requiring a Rust toolchain on the target machine.

What was the solution? (How)

Update the GitHub release workflow to use maturin to build wheels on each target platform (Mac, Windows, Linux, on both x86_64 and ARM architectures).

Reusable workflows are being added to the .github repo in this PR (will be reused in openjd-sessions as well): OpenJobDescription/.github#17

What is the impact of this change?

Starting from the next release, we will have platform specific wheels published to PyPI. pip install openjd-model will resolve a pre-built native wheel on all supported platforms. Users no longer need a Rust toolchain to install the package on Windows, macOS, or Linux.

How was this change tested?

Made a test branch that runs the multi platform build and downloads the artifacts and shows they are in the correct folder layout as expected: https://github.com/jericht/openjd-model-for-python/actions/runs/27854658792

  • Builds the wheel on all platforms
  • Prints out the resulting artifact layout (for consumption by reusable_release.yml)
  • Installs the wheel and imports both the Python module and Rust modules from Python on all platforms

Output from above run:

=== build-artifact layout ===
build-artifact/
└── dist
    ├── openjd_model-0.0.1-cp39-abi3-macosx_10_12_x86_64.whl
    ├── openjd_model-0.0.1-cp39-abi3-macosx_11_0_arm64.whl
    ├── openjd_model-0.0.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
    ├── openjd_model-0.0.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
    ├── openjd_model-0.0.1-cp39-abi3-win_amd64.whl
    ├── openjd_model-0.0.1-cp39-abi3-win_arm64.whl
    └── openjd_model-0.9.0.tar.gz

2 directories, 7 files

=== file details ===
total 27224
drwxr-xr-x 2 runner runner    4096 Jun 18 22:07 .
drwxr-xr-x 3 runner runner    4096 Jun 18 22:07 ..
-rw-r--r-- 1 runner runner 4704132 Jun 18 22:07 openjd_model-0.0.1-cp39-abi3-macosx_10_12_x86_64.whl
-rw-r--r-- 1 runner runner 4546002 Jun 18 22:07 openjd_model-0.0.1-cp39-abi3-macosx_11_0_arm64.whl
-rw-r--r-- 1 runner runner 4881124 Jun 18 22:07 openjd_model-0.0.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
-rw-r--r-- 1 runner runner 4939740 Jun 18 22:07 openjd_model-0.0.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
-rw-r--r-- 1 runner runner 4409145 Jun 18 22:07 openjd_model-0.0.1-cp39-abi3-win_amd64.whl
-rw-r--r-- 1 runner runner 4166130 Jun 18 22:07 openjd_model-0.0.1-cp39-abi3-win_arm64.whl
-rw-r--r-- 1 runner runner  209662 Jun 18 22:07 openjd_model-0.9.0.tar.gz

Example verification on Windows runner:

  python -c "
  from openjd.model._version import version
  print(f'openjd-model version: {version}')
  
  from openjd._openjd_rs import FormatString
  print(f'Native extension loaded: FormatString={FormatString}')
  
  from openjd.model import parse_model
  print('parse_model imported successfully')
  
  import platform
  print(f'Platform: {platform.system()} {platform.machine()}')
  print('ALL CHECKS PASSED')
  "
  shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
  env:
    pythonLocation: C:\hostedtoolcache\windows\Python\3.13.14\x64
    PKG_CONFIG_PATH: C:\hostedtoolcache\windows\Python\3.13.14\x64/lib/pkgconfig
    Python_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.13.14\x64
    Python2_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.13.14\x64
    Python3_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.13.14\x64
openjd-model version: 0.0.1
Native extension loaded: FormatString=<class 'openjd.expr.FormatString'>
parse_model imported successfully
Platform: Windows AMD64
ALL CHECKS PASSED

Was this change documented?

N/A

Is this a breaking change?

No.

Does this change impact security?

No


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

with:
tag: ${{ needs.TagRelease.outputs.tag }}
needs: [TagRelease, PreRelease]
uses: OpenJobDescription/.github/.github/workflows/reusable_release.yml@mainline

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the key line to call to the shared workflow right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually an existing release flow, the above one (reusable_maturin_prerelease) is the new reusable workflow added in OpenJobDescription/.github#17

@jericht
jericht marked this pull request as ready for review June 20, 2026 00:35
@jericht
jericht requested a review from a team as a code owner June 20, 2026 00:35
@jericht
jericht force-pushed the jericht/multi_platform_build branch from 332ca19 to 46847ca Compare June 20, 2026 01:40
Comment on lines -57 to -59
Publish:
needs: [TagRelease, Release]
uses: OpenJobDescription/.github/.github/workflows/reusable_publish_python.yml@mainline

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step is being removed because we are moving away from the CodeBuild based internal build system.

@@ -35,57 +35,36 @@ jobs:
tag: ${{ needs.TagRelease.outputs.tag }}

PreRelease:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, this is how I reviewed it. I asked my agent to look at the before and after, and sanity check the paths:

Here's the summary of the release workflow changes:
  
  Before (current mainline)
  
  The workflow pipeline is:
  
  1. TagRelease → creates a git tag
  2. UnitTests → runs code quality checks
  3. PreRelease → uses reusable_prerelease.yml (generic pre-release)
  4. Release → uses reusable_release.yml (creates the GitHub Release)
  5. Publish → uses reusable_publish_python.yml (a separate reusable workflow step)
  6. PublishToPyPI → checks out the tag, sets up Python 3.9, installs hatch, runs hatch -v build to build the package locally on a single Ubuntu runner, then publishes the resulting wheel/sdist to PyPI via trusted
  publishing
  
  The key point: the build happens at PyPI-publish time on a single Ubuntu runner using hatch build. This produces a single pure-Python py3-none-any wheel since there was no native code.
  
  After (this PR)
  
  The workflow pipeline becomes:
  
  1. TagRelease → same
  2. UnitTests → same
  3. PreRelease → now uses reusable_maturin_prerelease.yml — a new reusable workflow that builds platform-specific wheels via maturin across multiple OS/arch combinations (macOS x86_64/arm64, Linux x86_64/aarch64,
  Windows amd64/arm64). It also no longer requests id-token/contents: write permissions itself.
  4. Release → same reusable workflow (creates GitHub Release), now with explicit permissions
  5. Publish job is removed entirely
  6. PublishToPyPI → no longer checks out code or builds anything. Instead it downloads the pre-built build-artifact (the multi-platform wheels produced by PreRelease) and publishes them directly to PyPI.
  Dependencies changed from [TagRelease, Publish] to [TagRelease, Release, PreRelease].
  
  Other changes
  
  - _build_backend.py: On Windows, sets CARGO_TARGET_DIR=C:\cargo-target to avoid MSVC linker failures from long paths when building from an extracted sdist.
  - scripts/maturin_build.py: Adds a --version-only flag that patches version into pyproject.toml and _version.py without running maturin — used by CI when maturin-action handles the actual compilation but needs
  the version injected beforehand.
  

Comment thread .github/workflows/release_publish.yml
Replace the single-platform hatch build with a GitHub Actions matrix
that builds native wheels for linux (x86_64, aarch64), macOS (x86_64,
arm64), and Windows (x86_64, arm64) using maturin-action.

Add --version-only flag to scripts/maturin_build.py so CI can inject
the VCS version into pyproject.toml before maturin-action runs.

Signed-off-by: Jericho Tolentino <68654047+jericht@users.noreply.github.com>
@jericht
jericht force-pushed the jericht/multi_platform_build branch from 46847ca to 30a6fca Compare June 23, 2026 19:07
@jericht
jericht enabled auto-merge (squash) June 23, 2026 19:07
@jericht
jericht disabled auto-merge June 23, 2026 19:07
@jericht
jericht enabled auto-merge (squash) June 23, 2026 19:07
@jericht
jericht merged commit 1c30177 into OpenJobDescription:mainline Jun 23, 2026
51 of 57 checks passed
@jericht
jericht deleted the jericht/multi_platform_build branch June 23, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants