Modernize pelican project to PEP-standard pyproject.toml with uv and Hatch

[potiuk]

Summary

Modernizes the pelican/ subtree into a fully PEP-standard Python project,
with both uv (proprietary, astral-sh) and Hatch (PyPA/PSF-managed)
wired in as first-class development environment front-ends sharing a single
PEP 735 dependency group. Runtime installs (Dockerfile and the composite
action) are rewired onto uv tool install, and requirements.txt is gone —
pyproject.toml is now the single source of truth.

Both environment managers follow modern Python standards and are set up so a
contributor can pick whichever tool they prefer without losing anything:

• uv — proprietary but the current de-facto fast runner, backed by
  Astral. Used for the actual install path in action.yml and Dockerfile
  (uv tool install 'pelican[markdown]' --with .) and as one of the three
  documented local-dev flows (uv sync).
• Hatch — the PyPA/PSF-managed project manager. Configured via
  [tool.hatch.envs.default] with installer = "uv" and
  dependency-groups = ["dev"], so it resolves the same PEP 735 dev group
  that uv does.

Modern standards adopted

Standard	Where	Notes
PEP 517 / 518	[build-system]	setuptools backend, declared build requires
PEP 621	[project]	metadata, deps, classifiers, authors, urls
PEP 639	license = "Apache-2.0"	SPDX license expression
PEP 735	[dependency-groups].dev	single source of truth for dev tooling

What changed

• pyproject.toml (new) — PEP 621 metadata, SPDX Apache-2.0 license,
  requires-python = ">=3.10", curated runtime deps in
  [project].dependencies, a PEP 735 [dependency-groups].dev holding
  pelican[markdown]>=4.11,<4.12 + ruff + pytest, [tool.uv] with
  required-version = ">=0.5.0", [tool.hatch.envs.default] with
  dependency-groups = ["dev"] and installer = "uv", plus
  [tool.hatch.envs.default.scripts] named lint / fmt / test.
• requirements.txt (deleted) — pyproject.toml is now canonical.
• Dockerfile — bootstraps uv, runs
  uv tool install 'pelican[markdown]' --with . against the in-image
  checkout, and rebuilds the pelicanasf wrapper around the tool venv's
  pelican CLI. The wrapper resolves the venv's own Python at build time
  so user pelicanconf.py can import any dep that lives in the same venv.
• action.yml — adds actions/setup-python@v5 as the first step,
  bootstraps uv via pip install uv (no more PIP_BREAK_SYSTEM_PACKAGES),
  and runs
  uv tool install "pelican[markdown]==${{ inputs.version }}" --with ${{ github.action_path }}
  with optional --with-requirements ${{ inputs.requirements }} layered
  in. The Generate step now calls pelican directly and runs plugin_paths
  as a module through $PELICAN_TOOL_PY (published via GITHUB_ENV).
• README.md — full rewrite of the working-with-the-project sections:
  PEP 621-style inputs table, project layout table, three documented dev
  workflows (uv sync, manual uv venv, hatch env create), tool version
  floors (uv >=0.5.0, hatch >=1.16.0), and a plain-English explanation
  of the shared tool-venv execution model used by both the composite
  action and the Docker image.

Behaviour changes worth flagging

• The Docker image no longer pins pelican==4.5.4; it now follows the
  action's inputs.version default (4.11.0.post0), aligning the Docker
  path with the composite action path. The stale MarkupSafe<2.1.0
  workaround comment block tied to the 4.5.4 pin is gone.
• The composite action uses a hermetic Python provisioned by
  actions/setup-python@v5 (currently pinned to 3.12) rather than the
  runner's system-managed Python.

Test plan

Validated locally on macOS against the real pyproject.toml:

• uv tool install 'pelican[markdown]' --with . — resolves Pelican
  4.11.0.post0 + all runtime deps into a single tool venv; the
  pelican CLI is on PATH and python -m plugin_paths works from the
  tool venv's own Python.
• uv sync — creates .venv/ + uv.lock, pulls in the dev group
  by default, and python -c "import pelican, ruff, plugin_paths"
  succeeds.
• uv venv && uv pip install -e . && uv pip install --group dev —
  alternate manual flow validated.
• hatch env create / hatch run python -c "import pelican, ruff, plugin_paths" — Hatch reads [dependency-groups].dev via
  dependency-groups = ["dev"] (confirmed against Hatch 1.16.5 with a
  freshly recreated env).
• [tool.uv].required-version = ">=0.5.0" enforced — uv sync passes
  the check; parse-verified via tomllib.
• Smoke test the composite action on a real workflow (publish and
  build-only) — needs a PR-triggered run on a downstream pelican site.
• Build the Docker image end-to-end and render a sample site via
  pelicanasf content — not yet run locally (requires the full cmark
  build chain); will rely on CI / manual follow-up.

🤖 Generated with Claude Code

[dave2wave]

I will test tomorrow against some my uses of the action.

[Copilot]

Pull request overview

Modernizes the pelican/ subtree into a PEP-standard Python project (via pyproject.toml) and updates the composite action + Docker runtime to install Pelican and the action’s dependencies using uv tool install, removing requirements.txt as the dependency source of truth.

Changes:

• Replace requirements.txt-based dependency management with pelican/pyproject.toml (PEP 621 + PEP 735 dependency groups), plus Hatch/uv configuration.
• Rework pelican/action.yml to provision Python via actions/setup-python, install uv, and install Pelican + action deps into an isolated uv tool venv.
• Update pelican/Dockerfile to bootstrap uv and install Pelican + action deps via uv tool install, plus adjust the pelicanasf wrapper to run plugin_paths inside the tool venv.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pelican/requirements.txt	Removed in favor of pyproject.toml as the single dependency source of truth.
pelican/README.md	Rewritten docs covering inputs, layout, and uv/Hatch workflows.
pelican/pyproject.toml	New PEP-standard project metadata, runtime deps, PEP 735 dev group, and tooling config (uv/Hatch/ruff).
pelican/Dockerfile	Switches runtime install flow to uv tool install and updates the pelicanasf wrapper to use the tool venv’s Python for plugin_paths.
pelican/action.yml	Uses setup-python + uv tool installs; runs Pelican directly from the uv tool venv and executes plugin_paths via the tool venv Python.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sebbASF]

The dependabot change appears to be tangential to this PR.

I don't have any experience with setting up uv, so cannot comment otherwise

[potiuk]

Oh. absolutely. But the copilot has a very oudated approach - they way it locks dependencies it's not only few years old but it also does not allow for modern ways of keeping it up-to-date (while providing reproducibility).

But I am so glad that you asked @dave2wave and @sebbASF and that you care about reproducibility.

Actually when I prepared the pr, I had an option to add uv.lock to the the PR, I thought that it will be too much to add to the PR, but since reproducibility seems to be an important aspect, I added it.

Generally speaking uv supports very modern reproducibility flows with --frozen installation that will never go above the reproducibility run.

Actually - to be perfectly honest - what dependabot (and previous approach) proposed was not at all reproducible. At most it was "reproduce-ish". It only pinned one dependency, but all the transitive dependencies were not pinned - which did not guarantee python package reproducibility.

Uv lock, and the way how it pins not only the specified dependency, but all the transitive ones and also all the architecture variations for reproducibility, is the absolute modern standard for reproducibility - of both modern development environment and also it allows to add dependabot upgrades when security issues are published for them.

[potiuk]

The dependabot change appears to be tangential to this PR.

I don't have any experience with setting up uv, so cannot comment otherwise

Not really tangential - it's just how adding managing dependencies work in modern projects. Dependabot understandlats uv lock and pyproject.toml - and adding it to the modern way of packaging (especially with cooldown) allows few things:

• reproducible environment for development (until you update the uv.lock - either semi-automatically with dependabot or manually)
• cooldown delays installing latest dependencies while they migh have maliciously pushed versions (see trivy + LiteLLM recent problems)
• while they also allow to automatically keep the packages updated with latest security fixes (including transitive dependencies).

so - those are not really tangential changes, those are all changes to modernise general setup of dependencies, development environment and package management. This absolutely follows all the modern standards - published by Python Packaging Authority (PyPA) - some of them as old as 4-5 years. All modern tools and IDEs support those as well - for example opening such "pelican" folder in an IDE will result in a working version that you can run tests on and develop without any setup at all - just auto-discovery will have everything created. With uv and hatch you can basically even start developing on the project even without havin python version installed for development. With hatch or uv it will automatically download and install python version matching your pyproject.toml and you can develop wiht 0 env setup basically.

Python packaging and tooling had gone a long way over last 4 years - matching now all modern language development. It also makes it easier for agentic workflows - standard project setup allows agents to disover how to run tests and develop code easily - so for example they will run all the tests, formatting and linting automatically - based on pyproject.toml. Dependabot config is just one more automation that makes overhead for contibution or management of such project, generally no-op.

[potiuk]

• added uv.lock
• added detaiked documentation explaining the workflows - both automated and manual explaining how dependency management works.

[potiuk]

I also added nicer handling of the case when pelicanconf.py is not found:

[jarekpotiuk:~/code/infrastructure-actions/pelican] modernize-pelican-pyproject 2s ± docker run --rm -it -p 8000:8000 -v "$PWD":/site pelican-asf
pelicanasf: pelicanconf.py not found in /site.
Mount a Pelican website checkout (one containing pelicanconf.py) at /site, e.g.
  docker run --rm -it -p 8000:8000 -v "$PWD":/site <image>

[potiuk]

Also - it nicely runs pelican-test action in my fork https://github.com/potiuk/infrastructure-actions/actions/runs/24318747979/job/71001351532

[dave2wave]

This will close #92

[dave2wave]

I reality wish that this PR was a branch in the apache/infrastructure-actions repository. If it were I could TEST it. Unfortunately I cannot easily test that this has no regressions. Please have no fear of creating a PR in a branch of the main repository.

[potiuk]

I fear not: https://github.com/apache/infrastructure-actions/tree/modernize-pelican-pyproject -> branch created.

BTW. It's easy to push it by anyone:

gh co #702
git push origin modernize-pelican-pyproject 

Is what I did :) .

Change was made

[dave2wave]

I tested the build using apache/tooling-docs and the pages had no unexpected markup changes.

I will note that the action is not widely used:

1. datafusion-site
2. flex-site
3. petri
4. solr-site
5. tooling-docs

I'm responsible for two of these. We should warn datafusion, flex, and Solr.

[dave2wave]

@sebbASF Do you have any opinion about the Dockerfile?

[dave2wave]

Once we merge I'll let the three projects know so that they can pin to the old sha if they get unexpected behavior.

[potiuk]

Once we merge I'll let the three projects know so that they can pin to the old sha if they get unexpected behavior.

good idea.

[dave2wave]

I sent an email to users@infra bcc'd to the three PMC's dev lists.

[potiuk]

I sent an email to users@infra bcc'd to the three PMC's dev lists.

Cool. Let's see what they come back with :)