[libclang/python][ci] Add release Clang Python Bindings CI workflow #168234
+180
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a pyproject.toml file for creating sdist and wheel packages for the clang python bindings. It is required to move updates of the clang and libclang PyPI packages to the LLVM monorepo. Versioning information is derived from LLVM git tags, so no manual updates are needed to bump version numbers. The minimum python version required is set to 3.10 due to cindex.py using PEP 604 union type syntax (str | bytes | None). Signed-off-by: Ryan Mast <mast.ryan@gmail.com>
This migrates the CI workflow from https://github.com/trolldbois/python-clang/blob/master/.github/workflows/release.yml and adapts it to work within the structure of the LLVM monorepo. It builds upon the Python packaging files added in llvm#125806, and should be merged after that one. The CI workflow is set up to use two jobs: - the first just builds the package and saves it as an artifact and will work for testing in forks, PRs, and untagged commits - the second checks for release manager permissions and uses trusted publishing to upload the package to PyPI Signed-off-by: Ryan Mast <mast.ryan@gmail.com>
llvmbot
added
clang
Member
|
@llvm/pr-subscribers-github-workflow Author: Ryan Mast (nightlark) ChangesThis migrates the CI workflow from https://github.com/trolldbois/python-clang/blob/master/.github/workflows/release.yml and adapts it to work within the structure of the LLVM monorepo. It builds upon the Python packaging files added in #125806, and should be merged after that one. The CI workflow is set up to use two jobs:
Full diff: https://github.com/llvm/llvm-project/pull/168234.diff 5 Files Affected:
diff --git a/.gitattributes b/.gitattributes
index 6b281f33f737d..142d6689f1088 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -1,3 +1,5 @@
+clang/bindings/python/.git_archival.txt export-subst
+
libcxx/src/**/*.cpp merge=libcxx-reformat
libcxx/include/**/*.h merge=libcxx-reformat
diff --git a/.github/workflows/release-clang-pypi.yml b/.github/workflows/release-clang-pypi.yml
new file mode 100644
index 0000000000000..8c0b00a574cf5
--- /dev/null
+++ b/.github/workflows/release-clang-pypi.yml
@@ -0,0 +1,111 @@
+name: Release Clang Python Bindings
+
+permissions:
+ contents: read
+
+on:
+ push:
+ branches:
+ - main
+ - release/*
+ paths:
+ - .github/workflows/release-clang-pypi.yml
+ - 'clang/bindings/python/**'
+
+ pull_request:
+ paths:
+ - .github/workflows/release-clang-pypi.yml
+ - 'clang/bindings/python/**'
+
+ workflow_dispatch:
+ inputs:
+ release-version:
+ description: 'Release Version'
+ required: false
+ type: string
+
+ workflow_call:
+ inputs:
+ release-version:
+ description: 'Release Version'
+ required: true
+ type: string
+ secrets:
+ RELEASE_TASKS_USER_TOKEN:
+ description: "Secret used to check user permissions."
+ required: false
+
+jobs:
+ build-release:
+ if: github.repository_owner == 'llvm' || github.event_name == 'workflow_dispatch'
+ runs-on: ubuntu-24.04
+ steps:
+ - name: Checkout LLVM (tagged release)
+ if: inputs.release-version != ''
+ uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+ with:
+ ref: "llvmorg-${{ inputs.release-version }}"
+ fetch-depth: 0
+ sparse-checkout: clang/bindings/python/
+
+ - name: Checkout LLVM (latest commit)
+ if: inputs.release-version == ''
+ uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+ with:
+ fetch-depth: 0
+ sparse-checkout: clang/bindings/python/
+
+ - uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
+ with:
+ python-version: '3.13'
+
+ - name: Package Clang Python Bindings
+ working-directory: clang/bindings/python
+ run: |
+ pip install build twine
+ python -m build
+ python -m twine check dist/*
+
+ - name: Upload Clang Python Bindings package dist artifacts
+ uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+ with:
+ name: python-package-dist
+ path: clang/bindings/python/dist
+
+ pypi-publish:
+ name: Upload release to PyPI
+ runs-on: ubuntu-24.04
+ needs: build-release
+ # Only run this job in the main llvm repository when a release version is provided.
+ if: github.repository_owner == 'llvm' && inputs.release-version != ''
+ environment:
+ name: pypi
+ url: https://pypi.org/p/clang
+ permissions:
+ id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
+ steps:
+ - name: Checkout github-upload-release.py for permissions check
+ uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+ with:
+ sparse-checkout: llvm/utils/release/github-upload-release.py
+
+ - name: Install dependencies for github-upload-release.py
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y python3-github
+
+ - name: Check Permissions
+ env:
+ GITHUB_TOKEN: ${{ github.token }}
+ USER_TOKEN: ${{ secrets.RELEASE_TASKS_USER_TOKEN }}
+ run: |
+ ./llvm/utils/release/github-upload-release.py --token "$GITHUB_TOKEN" --user ${{ github.actor }} --user-token "$USER_TOKEN" check-permissions
+
+ - name: Download Python package dist artifacts
+ uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
+ with:
+ name: python-package-dist
+ path: dist
+
+ - name: Publish package distributions to PyPI
+ uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
\ No newline at end of file
diff --git a/clang/bindings/python/.git_archival.txt b/clang/bindings/python/.git_archival.txt
new file mode 100644
index 0000000000000..7876d4af4c620
--- /dev/null
+++ b/clang/bindings/python/.git_archival.txt
@@ -0,0 +1,3 @@
+node: $Format:%H$
+node-date: $Format:%cI$
+describe-name: $Format:%(describe:tags=true,match=llvmorg-*[0-9]*)$
diff --git a/clang/bindings/python/.gitignore b/clang/bindings/python/.gitignore
new file mode 100644
index 0000000000000..1641a745fb682
--- /dev/null
+++ b/clang/bindings/python/.gitignore
@@ -0,0 +1,21 @@
+# setuptools_scm auto-generated version file
+_version.py
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Distribution / packaging
+build/
+dist/
+*.egg-info/
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
diff --git a/clang/bindings/python/pyproject.toml b/clang/bindings/python/pyproject.toml
new file mode 100644
index 0000000000000..69fd628e8222a
--- /dev/null
+++ b/clang/bindings/python/pyproject.toml
@@ -0,0 +1,43 @@
+[build-system]
+requires = ["hatchling>=1.27", "hatch-vcs>=0.4"]
+build-backend = "hatchling.build"
+
+[project]
+name = "clang"
+description = "clang python bindings"
+readme = {file = "README.txt", content-type = "text/plain"}
+
+license = "Apache-2.0 WITH LLVM-exception"
+authors = [
+ { name = "LLVM" }
+]
+keywords = ["llvm", "clang", "libclang"]
+classifiers = [
+ "Intended Audience :: Developers",
+ "Development Status :: 5 - Production/Stable",
+ "Topic :: Software Development :: Compilers",
+ "Operating System :: OS Independent",
+ "Programming Language :: Python :: 3",
+]
+requires-python = ">=3.10"
+dynamic = ["version"]
+
+[project.urls]
+Homepage = "https://clang.llvm.org/"
+Download = "https://llvm.org/releases/download.html"
+Discussions = "https://discourse.llvm.org/"
+"Issue Tracker" = "https://github.com/llvm/llvm-project/issues"
+"Source Code" = "https://github.com/llvm/llvm-project/tree/main/clang/bindings/python"
+
+[tool.hatch.version]
+source = "vcs"
+version-scheme = "no-guess-dev"
+# regex version capture group gets x.y.z with optional -rcN, -aN, -bN suffixes; -init is just consumed
+tag-pattern = "^llvmorg-(?P<version>\\d+(?:\\.\\d+)*(?:-rc\\d+)?)"
+
+[tool.hatch.build.hooks.vcs]
+version-file = "clang/_version.py"
+
+[tool.hatch.version.raw-options]
+search_parent_directories = true
+version_scheme = "no-guess-dev"
|
Member
|
@llvm/pr-subscribers-clang Author: Ryan Mast (nightlark) ChangesThis migrates the CI workflow from https://github.com/trolldbois/python-clang/blob/master/.github/workflows/release.yml and adapts it to work within the structure of the LLVM monorepo. It builds upon the Python packaging files added in #125806, and should be merged after that one. The CI workflow is set up to use two jobs:
Full diff: https://github.com/llvm/llvm-project/pull/168234.diff 5 Files Affected:
diff --git a/.gitattributes b/.gitattributes
index 6b281f33f737d..142d6689f1088 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -1,3 +1,5 @@
+clang/bindings/python/.git_archival.txt export-subst
+
libcxx/src/**/*.cpp merge=libcxx-reformat
libcxx/include/**/*.h merge=libcxx-reformat
diff --git a/.github/workflows/release-clang-pypi.yml b/.github/workflows/release-clang-pypi.yml
new file mode 100644
index 0000000000000..8c0b00a574cf5
--- /dev/null
+++ b/.github/workflows/release-clang-pypi.yml
@@ -0,0 +1,111 @@
+name: Release Clang Python Bindings
+
+permissions:
+ contents: read
+
+on:
+ push:
+ branches:
+ - main
+ - release/*
+ paths:
+ - .github/workflows/release-clang-pypi.yml
+ - 'clang/bindings/python/**'
+
+ pull_request:
+ paths:
+ - .github/workflows/release-clang-pypi.yml
+ - 'clang/bindings/python/**'
+
+ workflow_dispatch:
+ inputs:
+ release-version:
+ description: 'Release Version'
+ required: false
+ type: string
+
+ workflow_call:
+ inputs:
+ release-version:
+ description: 'Release Version'
+ required: true
+ type: string
+ secrets:
+ RELEASE_TASKS_USER_TOKEN:
+ description: "Secret used to check user permissions."
+ required: false
+
+jobs:
+ build-release:
+ if: github.repository_owner == 'llvm' || github.event_name == 'workflow_dispatch'
+ runs-on: ubuntu-24.04
+ steps:
+ - name: Checkout LLVM (tagged release)
+ if: inputs.release-version != ''
+ uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+ with:
+ ref: "llvmorg-${{ inputs.release-version }}"
+ fetch-depth: 0
+ sparse-checkout: clang/bindings/python/
+
+ - name: Checkout LLVM (latest commit)
+ if: inputs.release-version == ''
+ uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+ with:
+ fetch-depth: 0
+ sparse-checkout: clang/bindings/python/
+
+ - uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
+ with:
+ python-version: '3.13'
+
+ - name: Package Clang Python Bindings
+ working-directory: clang/bindings/python
+ run: |
+ pip install build twine
+ python -m build
+ python -m twine check dist/*
+
+ - name: Upload Clang Python Bindings package dist artifacts
+ uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+ with:
+ name: python-package-dist
+ path: clang/bindings/python/dist
+
+ pypi-publish:
+ name: Upload release to PyPI
+ runs-on: ubuntu-24.04
+ needs: build-release
+ # Only run this job in the main llvm repository when a release version is provided.
+ if: github.repository_owner == 'llvm' && inputs.release-version != ''
+ environment:
+ name: pypi
+ url: https://pypi.org/p/clang
+ permissions:
+ id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
+ steps:
+ - name: Checkout github-upload-release.py for permissions check
+ uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+ with:
+ sparse-checkout: llvm/utils/release/github-upload-release.py
+
+ - name: Install dependencies for github-upload-release.py
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y python3-github
+
+ - name: Check Permissions
+ env:
+ GITHUB_TOKEN: ${{ github.token }}
+ USER_TOKEN: ${{ secrets.RELEASE_TASKS_USER_TOKEN }}
+ run: |
+ ./llvm/utils/release/github-upload-release.py --token "$GITHUB_TOKEN" --user ${{ github.actor }} --user-token "$USER_TOKEN" check-permissions
+
+ - name: Download Python package dist artifacts
+ uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
+ with:
+ name: python-package-dist
+ path: dist
+
+ - name: Publish package distributions to PyPI
+ uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
\ No newline at end of file
diff --git a/clang/bindings/python/.git_archival.txt b/clang/bindings/python/.git_archival.txt
new file mode 100644
index 0000000000000..7876d4af4c620
--- /dev/null
+++ b/clang/bindings/python/.git_archival.txt
@@ -0,0 +1,3 @@
+node: $Format:%H$
+node-date: $Format:%cI$
+describe-name: $Format:%(describe:tags=true,match=llvmorg-*[0-9]*)$
diff --git a/clang/bindings/python/.gitignore b/clang/bindings/python/.gitignore
new file mode 100644
index 0000000000000..1641a745fb682
--- /dev/null
+++ b/clang/bindings/python/.gitignore
@@ -0,0 +1,21 @@
+# setuptools_scm auto-generated version file
+_version.py
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Distribution / packaging
+build/
+dist/
+*.egg-info/
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
diff --git a/clang/bindings/python/pyproject.toml b/clang/bindings/python/pyproject.toml
new file mode 100644
index 0000000000000..69fd628e8222a
--- /dev/null
+++ b/clang/bindings/python/pyproject.toml
@@ -0,0 +1,43 @@
+[build-system]
+requires = ["hatchling>=1.27", "hatch-vcs>=0.4"]
+build-backend = "hatchling.build"
+
+[project]
+name = "clang"
+description = "clang python bindings"
+readme = {file = "README.txt", content-type = "text/plain"}
+
+license = "Apache-2.0 WITH LLVM-exception"
+authors = [
+ { name = "LLVM" }
+]
+keywords = ["llvm", "clang", "libclang"]
+classifiers = [
+ "Intended Audience :: Developers",
+ "Development Status :: 5 - Production/Stable",
+ "Topic :: Software Development :: Compilers",
+ "Operating System :: OS Independent",
+ "Programming Language :: Python :: 3",
+]
+requires-python = ">=3.10"
+dynamic = ["version"]
+
+[project.urls]
+Homepage = "https://clang.llvm.org/"
+Download = "https://llvm.org/releases/download.html"
+Discussions = "https://discourse.llvm.org/"
+"Issue Tracker" = "https://github.com/llvm/llvm-project/issues"
+"Source Code" = "https://github.com/llvm/llvm-project/tree/main/clang/bindings/python"
+
+[tool.hatch.version]
+source = "vcs"
+version-scheme = "no-guess-dev"
+# regex version capture group gets x.y.z with optional -rcN, -aN, -bN suffixes; -init is just consumed
+tag-pattern = "^llvmorg-(?P<version>\\d+(?:\\.\\d+)*(?:-rc\\d+)?)"
+
+[tool.hatch.build.hooks.vcs]
+version-file = "clang/_version.py"
+
+[tool.hatch.version.raw-options]
+search_parent_directories = true
+version_scheme = "no-guess-dev"
|
5 tasks
Contributor
Author
|
Here are some test runs of the workflow that I did in my fork:
|
Contributor
DeinAlptraum
left a comment
DeinAlptraum
left a comment
There was a problem hiding this comment.
Thanks a lot for your work on this! I highly appreciate you getting the bindings releases into a better shape.
As mentioned before, I'm not familiar with Python packaging so I can't contribute too much here unfortunately, Insofar as I can tell though, this LGTM
|
|
||
| [project.urls] | ||
| Homepage = "https://clang.llvm.org/" | ||
| Download = "https://llvm.org/releases/download.html" |
Contributor
There was a problem hiding this comment.
This website seems super outdated, are you sure there isn't a better one?
Contributor
Author
There was a problem hiding this comment.
For the homepage maybe the website theme is a bit on the dated side? Looking at the C++ status page, it lists features that were added in the v21 release: https://clang.llvm.org/cxx_status.html
The downloads page looks like it could use updating… or perhaps it could just get pointed at the GitHub releases page for LLVM?
Comment on lines
+11
to
+18
| paths: | ||
| - .github/workflows/release-clang-pypi.yml | ||
| - 'clang/bindings/python/**' | ||
|
|
||
| pull_request: | ||
| paths: | ||
| - .github/workflows/release-clang-pypi.yml | ||
| - 'clang/bindings/python/**' |
Contributor
There was a problem hiding this comment.
Perhaps the paths here should also include the underlying C/C++ sources? I.e.
'clang/tools/libclang/**'
as we currently also do in the test CI workflow:
https://github.com/llvm/llvm-project/blob/main/.github/workflows/libclang-python-tests.yml
Contributor
Author
There was a problem hiding this comment.
We could trigger for the underlying C/C++ sources, but for this stage of the packaging process it’s a pure Python wheel with just the Python bindings code and no compiled library, so changes to those files shouldn’t have any effect on the resulting packages.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This migrates the CI workflow from https://github.com/trolldbois/python-clang/blob/master/.github/workflows/release.yml and adapts it to work within the structure of the LLVM monorepo.
It builds upon the Python packaging files added in #125806, and should be merged after that one.
The CI workflow is set up to use two jobs: