Get provider registry build off the docs publish critical path

[shahar1]

Human Summary

Since the providers registry build was introduced into the docs release process, the release
process for the providers' docs became longer by about 20-45 minutes (median ~23).
The big issue is that this process is composed of sequential runs of different GitHub Actions
workflows in different repositories (airflow, airflow-site, and airflow-site-archive), and this
process shouldn't be interrupted. More run time means:

1. More time spent on watching the entire process.
2. More chance that things will go wrong in the critical path, and we might need to do things all
   over again.

In my opinion, the provider registry is not as critical a component as the docs - which means that
it would be better separated into another workflow that will be initiated separately.

related: #50648, #45641, #44373, #32491, #50570, #66436

Detailed investigation - measurements, code references and two open questions (AI-generated)

During a provider release wave the release manager runs breeze workflow-run publish-docs and has
to keep the terminal open and the machine awake until it returns — the whole chain is driven from
the release manager's laptop, hence the warning at
dev/README_RELEASE_PROVIDERS.md:1769-1772:

Make sure to NOT close the terminal while the command is running and to keep your computer from
sleep. This command will run several (3!) workflows from your terminal and this is important to
keep it running until completion.

On the 2026-07-22 wave the publish-docs-to-s3.yml run alone took 90 minutes, and the docs had
been live on S3 since minute 47. The rest was the provider registry build, which the release
manager gains nothing by waiting for. Two questions below about getting it off that path.

Measured

Per-job timings, three recent wave runs:

Job	30335779728	30042123097	29107680148
Build Info	0.7m	0.7m	0.7m
Build documentation	41.0m	34.2m	41.8m
Publish documentation to S3	4.3m	4.7m	4.5m
— docs are live from here —
Registry: Build CI image	32.5m	13.3m	31.8m
Registry: Build & Publish	10.5m	9.3m	9.7m
Total (wall clock)	90.2m	62.4m	88.9m
of which after docs were live	43.4m (48%)	22.7m (36%)	41.6m (47%)

(Total is run createdAt→updatedAt, so it exceeds the sum of the jobs by the queue gaps.
Build Java SDK Javadoc — 0.2m, parallel with Build documentation — is omitted.)

Over the last 20 runs of the workflow, excluding one failed and one cancelled run, split on whether
the Update Provider Registry job ran or was skipped:

Cohort	n	Median total	Range
Registry ran	11	59.9m	34.7–90.2m
Registry skipped	7	21.2m	14.4–22.1m

Not a clean A/B — the skipped runs are core version publishes plus one dev-branch run, whose
Build documentation median is 16.1m against 32.6m for the registry-ran cohort. Of the 38.7m
difference, roughly 23m is the registry and roughly 16m the larger docs build.

The registry sub-workflow on its own: median 23.0m, range 17.3–43.0m (n=11) — ~38% of the
median run, and 100% additive because it is strictly serial after the S3 publish. Over half of it
(median 13.4m, range 8.0–33.1m) is spent building a CI image before breeze registry extract-data
runs.

Question 1: why does the registry job build its own CI image?

registry-build.yml:59-94
calls ci-image-build.yml with branch: "main" and push-image: "false". branch only sets
DEFAULT_BRANCH — the image name and therefore the buildx cache source
(ci-image-build.yml:115).
No checkout-ref is passed, so
:125-129
checks out the workflow ref, i.e. the release tag. The job therefore builds the release tag's
sources under main's image name, against main's registry cache. The hit rate is unstable as a
result: the same tag rebuilt on 2026-07-23 and 2026-07-28 took 13.3m and 32.5m, the slow run
re-running install_os_dependencies.sh for 19.6m after #70285 landed on main.

Meanwhile
publish-docs-to-s3.yml:301-318
builds a CI image from the same ref, in the same run, and never stashes it. So every wave pays
for two full CI image builds from the same sources — the docs job's python3.10
(:101)
and the registry's python3.12
(registry-build.yml:88).

Could the registry job consume the docs job's image instead of rebuilding — accepting that this
would move registry extraction to 3.10 — or, if main-tip dependencies are acceptable, pull
ghcr.io/apache/airflow/main/ci/python3.12, which canary already pushes via
finalize-tests.yml → push-image-cache.yml? (For the pull variant note that
build-and-publish-registry sets its own
permissions: contents: read,
which replaces the workflow-level default, so it would need packages: read adding — though the
images are public.)

To be clear about what cannot be dropped: two of the three extractors need the image.
breeze registry extract-data runs extract_metadata.py, extract_parameters.py and
extract_connections.py
(registry_commands.py:112-117),
and module discovery uses runtime inspection inside Breeze where all providers are installed —
registry_commands.py:103-105 even pip installs suspended providers first. The question is
build-vs-reuse, not image-vs-no-image. It is worth answering regardless of question 2 — though
note the two interact: reusing the docs job's image assumes the current serialisation, so if
question 2 goes the parallel way, only the ghcr-pull variant survives.

Question 2: does update-registry need needs: [publish-docs-to-s3]?

publish-docs-to-s3.yml:609
is needs: [publish-docs-to-s3, build-info]. Dropping the first element — keeping
needs: [build-info] — would run the registry in parallel with the docs build. It stays a
workflow_call, so nothing changes about permissions, and build-info's destination /
registry-providers / registry-full-build outputs are still available — update-registry
consumes nothing from the publish-docs-to-s3 job. Recomputing the same 11 runs from their
per-job timings, the median run drops from 59.9m to ~39m and no run gets longer, though on 5 of
the 11 the registry is the longer branch and becomes the critical path itself. That is arithmetic
on historical timestamps, not an executed run — in particular it assumes the two CI-image builds
can run concurrently without contending for runners, which the timestamps cannot tell us.

The registry reads nothing the docs publish produces — its Sphinx objects.inv fetch goes to
apache-airflow-docs (eu-central-1), the main-branch canary bucket written by
ci-image-checks.yml:465,
not the live-docs-airflow-apache-org bucket the publish job writes. The two also write disjoint
prefixes (…/docs/ vs …/registry/), so there is no clobbering.

What it emits is the real question. The docs URLs the wave build generates are all /stable/:

# dev/registry/extract_metadata.py:63
AIRFLOW_PROVIDER_DOCS_URL = "https://airflow.apache.org/docs/{package_name}/stable/"
# dev/registry/extract_parameters.py:451
base_docs_url = f"https://airflow.apache.org/docs/apache-airflow-providers-{provider_id}/stable"

and they are used for deep links, not just landing pages — per-class API anchors
(extract_parameters.py:463-468) and per-connection pages (extract_metadata.py:317-326). Since
the inventory comes from the canary bucket, it already knows about modules newer than the release.
So publishing the registry before the docs gives a window in which /stable/ links point at the
previous version, and anything new in this release — new provider, new module, new connection
page — 404s until the docs land.

Recomputing that window over the same 11 runs (registry starting after build-info instead of
after the publish): in 5 of 11 the window is negative — the registry would still have landed
after the docs, so no exposure at all. On the other 6 it runs up to ~21 minutes, median ~3 minutes
across all 11. Note the shape: the largest windows (17–21m) are the waves where the registry's
CI-image build hit cache and took 8–13m, while the two 90-minute waves — where that build took
~32m — come out under 5 minutes. Fixing question 1 would make this window wider, not narrower.

There is a second consequence that is not just ordering: needs: [publish-docs-to-s3] currently
gates the registry, so a failed docs build or publish skips it. Remove the edge and the registry
publishes to live regardless, and the window stops being bounded — the registry would sit there
pointing at /stable/ pages that never landed. If we do this, the gate probably has to be
reinstated somewhere, e.g. on the S3 sync step rather than on the whole job.

Note that the GitHub source URLs are version-pinned (extract_metadata.py:64-66,
extract_parameters.py:450-452) and gated on the providers-<id>/<ver> tag existing
(find_latest_released_version, extract_metadata.py:507-521),
so the tag is a hard input — but the tag is pushed one step before docs publishing
(README_RELEASE_PROVIDERS.md:1706),
so it is already satisfied by the time the workflow starts.

So: is a stale window of that size on new modules acceptable in exchange for taking the registry
off the critical path, and where should the failed-publish gate live? If the window is not
acceptable, is there a cheaper ordering than "wait for the whole publish job"?

Worth recording why the fully decoupled variant is harder, in case someone proposes it: a
gh workflow run issued from inside Actions arrives as workflow_dispatch with
github.event.sender.login == github-actions[bot], which fails the committer allowlist at
registry-build.yml:65-81;
build-and-publish-registry needs that job, so the run would go green having done nothing — the
same silent-skip class as #65188. That would need a bot entry in the allowlist, which is what
#69460 proposes to delete. Neither of the questions above requires going there.

Separately

The release manager's wait is also longer than the workflow for an unrelated reason:
monitor_workflow_run
polls run-level status, so the apache/airflow-site refresh does not start until the whole run
finishes, registry included. Returning once Publish documentation to S3 succeeds would unblock the
terminal at ~47m instead of ~90m. The trade-off is failure visibility: today a failed run exits
non-zero in the release manager's terminal, and #65188 is a case of a registry problem going
unnoticed for weeks, so an early return needs a real signal rather than a printed URL. Fixing
question 2 removes most of the need for this.

Why now, given #50648

#50648 asked for faster docs publishing and was
closed on 2025-06-09 with:

Yeah. I think release date is more important than speed of publishing. Especially that it is not
that slow and with the "publish-docs" breeze command it's fine to just submit it in terminal
and let it run until it gets completed..

Closing

I am not reopening the artifact-reuse idea declined there — the release-date argument against it
still holds. But that assessment predates the registry, merged in
#62261 on 2026-03-07. #50648's body cites "12-15
minutes"; runs where the registry job fires now median 59.9m. "Submit it in terminal and let it
run" is a different proposition at 90 minutes than at 15.

Related

• Add Apache Airflow Provider Registry #62261 added the registry and the needs: chaining.
• publish-docs-to-s3 fails on tags cut before #66100 #66436 (release-breaking), Fix publish-docs Update Provider Registry job for non-provider distributions #65506 (surfaced a red check on every airflow-ctl / task-sdk docs
  publish) and Fix provider registry never updating after docs publish #65188 (the registry job silently never ran) are three bugs in the registry-input
  derivation inside this workflow's build-info job. Neither question above addresses that class.
• Gate release-process and secret-using workflows via a protected "release" deployment environment #69460 — replace the per-workflow committer allowlists with a protected release environment.
  It does not enumerate registry-build.yml, but the intent covers it.

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

---

Drafted-by: Claude Code (Opus 5); reviewed by @shahar1 before posting