Skip to content

Publish palace-util and palace-opds to PyPI on release (PP-4076)#3238

Draft
jonathangreen wants to merge 4 commits intomainfrom
chore/publish-workspace-packages
Draft

Publish palace-util and palace-opds to PyPI on release (PP-4076)#3238
jonathangreen wants to merge 4 commits intomainfrom
chore/publish-workspace-packages

Conversation

@jonathangreen
Copy link
Copy Markdown
Member

@jonathangreen jonathangreen commented Apr 14, 2026

Description

Two things:

  1. Dynamic versioning for both workspace packages. Each package's pyproject.toml now declares dynamic = ["version"] and sources the version from a committed _version.py stub via hatchling's built-in regex version source — no plugins. The stub carries a dev placeholder (0.0.0.dev0, __commit__ = None, __branch__ = None) so uv build --package … works out-of-box for local and CI builds without an explicit version override. The release workflow overwrites _version.py with real dunamai-computed values right before uv build, so the wheel metadata picks up the real version.

  2. publish-pypi.yml workflow triggered by published GitHub Releases. Matrix-builds both packages in parallel, uses PyPI's Trusted Publishing (OIDC) flow — no API tokens stored in the repo or in GitHub secrets. The workflow also pins the intra-workspace dep: uv build leaves workspace deps as bare palace-util in Requires-Dist, which would fail to resolve on PyPI, so the workflow seds in palace-util==<version> before building palace-opds.

Motivation and Context

With palace-util and palace-opds extracted as standalone packages, they should be installable from PyPI so downstream Palace services (and the wider OPDS-consuming community for palace-opds) can depend on them without pulling in the palace-manager monorepo. palace-manager itself is intentionally NOT published here — it needs additional work first (alembic and friends prevent a clean wheel build).

One-time PyPI setup

For each of the two projects (do once per project):

  1. Reserve the name on PyPI (manual first upload or admin assistance).
  2. In the PyPI project's Publishing settings, add a trusted publisher:
    • Owner: ThePalaceProject
    • Repository: circulation
    • Workflow filename: publish-pypi.yml
    • Environment name: pypi
  3. (Optional) Create the pypi environment in the repo's Settings → Environments and add an approval gate / branch restriction for an extra guard.

After that, publishing a GitHub Release with a tag that dunamai can parse as semver (e.g. v1.0.0) will automatically upload both wheels.

How Has This Been Tested?

  • Local uv build --package palace-util produces palace_util-0.0.0.dev0-py3-none-any.whl.
  • Local uv build --package palace-opds produces palace_opds-0.0.0.dev0-py3-none-any.whl.
  • Simulated CI flow (overwrote _version.py with __version__ = "1.2.3", then rebuilt) — wheel correctly tagged palace_util-1.2.3-py3-none-any.whl. Reverted the stub afterward.
  • Runtime: python -c "import palace.util; print(palace.util.__version__)"0.0.0.dev0.
  • Runtime: python -c "import palace.opds; print(palace.opds.__version__)"0.0.0.dev0.
  • mypy passes (1128 source files).
  • pytest tests/palace_util tests/palace_opds — 232 tests pass.

First real end-to-end verification happens on the first tagged release after this lands.

Checklist

  • I have updated the documentation accordingly.
  • All new and existing tests passed.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 14, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.30%. Comparing base (11171c1) to head (8d9177c).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3238   +/-   ##
=======================================
  Coverage   93.30%   93.30%           
=======================================
  Files         498      502    +4     
  Lines       46147    46157   +10     
  Branches     6318     6318           
=======================================
+ Hits        43059    43069   +10     
  Misses       2003     2003           
  Partials     1085     1085           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jonathangreen jonathangreen force-pushed the chore/extract-palace-opds branch 2 times, most recently from f42c076 to 568b49d Compare April 14, 2026 19:30
Base automatically changed from chore/extract-palace-opds to main April 14, 2026 19:40
@jonathangreen jonathangreen changed the title Publish palace-util and palace-opds to PyPI on release Publish palace-util and palace-opds to PyPI on release (PP-4076) Apr 14, 2026
Both packages now declare `dynamic = ["version"]` and source the
version from a committed `_version.py` stub via hatchling's regex
version source. The stub carries a dev placeholder (0.0.0.dev0) so
local / developer builds succeed out-of-box; the PyPI publish workflow
overwrites `_version.py` with the dunamai-computed release version
right before `uv build` to get the real version into the wheel
metadata.

`_version.py` also carries `__commit__` and `__branch__` (None on dev,
populated in CI) to match the shape of palace-manager's `_version.py`
— useful for `import palace.util; palace.util.__commit__` style
runtime lookup. Each package's `__init__.py` re-exports the three
attributes; since `_version.py` is committed, no try/except fallback
is needed (palace-manager keeps its try/except only because its
`_version.py` is generated at Docker build time and never committed).

palace-manager's own versioning is unchanged; it's not yet
PyPI-publishable for unrelated reasons.
Add a GitHub Actions workflow that builds and uploads both workspace
packages to PyPI when a GitHub Release is published, using PyPI's
Trusted Publishing (OIDC) flow — no API tokens stored.

The workflow:

- Runs a matrix job per package on parallel runners.
- Computes version / commit / branch from the release tag via dunamai.
- Overwrites the package's _version.py with the computed values so the
  wheel metadata picks up the release version (see the previous
  commit for why _version.py is the source of truth).
- Pins the intra-workspace dependency before building palace-opds:
  uv build leaves workspace deps as bare `palace-util` in the wheel's
  Requires-Dist, which would fail to resolve on PyPI; we sed in
  `palace-util==<version>` so consumers installing palace-opds get the
  matching palace-util.
- Uploads via pypa/gh-action-pypi-publish@release/v1 with OIDC.

One-time setup required on PyPI (see comment in the workflow file):
create the palace-util and palace-opds projects and register this
workflow + repo + `pypi` environment as a trusted publisher for each.

palace-manager is intentionally NOT published here — it needs more
work first (alembic etc. prevent a clean wheel build).
[tool.hatch.version] only needs the path — source = 'regex' is the
default, and hatch's default pattern matches plain __version__ = "…"
assignments. Drop the type annotation from __version__ in _version.py
(keep them on __commit__ / __branch__ where they still convey useful
optionality) so the default pattern matches. Update the publish
workflow to emit the same unannotated __version__ line.
uv add --package palace-opds 'palace-util==X.Y.Z' rewrites the
dependency spec in pyproject.toml without relying on a literal text
match (sed would break if the dep line ever changes formatting).
It resolves against the workspace source, so the pinned version does
not need to exist on PyPI yet — important for the publish step, where
the version is being uploaded now.
@jonathangreen jonathangreen force-pushed the chore/publish-workspace-packages branch from f1f8058 to 8d9177c Compare April 14, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant