Speed up Publish Docs to S3 with main CI image cache#65108
Merged
potiuk merged 1 commit intoapache:mainfrom Apr 13, 2026
Merged
Speed up Publish Docs to S3 with main CI image cache#65108potiuk merged 1 commit intoapache:mainfrom
potiuk merged 1 commit intoapache:mainfrom
Conversation
The `build-docs` job in `publish-docs-to-s3.yml` was rebuilding the CI image from scratch because it never logged in to ghcr.io, so Buildx could not reach the registry cache tag pushed by the main "Test" workflow. The manual `docker buildx build` fallback also had no `--cache-from`, and a hardcoded `python3.9` tag that no longer matched the Python version used for the docs build. Add a `docker login ghcr.io` step, set `DOCKER_CACHE=registry` and `PYTHON_MAJOR_MINOR_VERSION` explicitly, and grant `packages: read` on the job so `breeze ci-image build` consumes the same `<image>:cache-<platform>` tag that `push-image-cache.yml` publishes from main. Fix the stale python tag in the fallback and refresh the comments that still claimed the job "will not use cache".
potiuk
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496,
jscheffl and
kaxil
as code owners
boring-cyborg
bot
added
area:dev-tools
backport-to-v3-2-test
Member
Author
|
cc: @gopidesupavan @kaxil @vatsrahul1001 @vincbeck @eladkal - if you wondered why building docs for publishing takes extra few minutes - this is why it was. |
vincbeck
approved these changes
Apr 13, 2026
github-actions bot
pushed a commit
that referenced
this pull request
Apr 13, 2026
…ache (#65108) The `build-docs` job in `publish-docs-to-s3.yml` was rebuilding the CI image from scratch because it never logged in to ghcr.io, so Buildx could not reach the registry cache tag pushed by the main "Test" workflow. The manual `docker buildx build` fallback also had no `--cache-from`, and a hardcoded `python3.9` tag that no longer matched the Python version used for the docs build. Add a `docker login ghcr.io` step, set `DOCKER_CACHE=registry` and `PYTHON_MAJOR_MINOR_VERSION` explicitly, and grant `packages: read` on the job so `breeze ci-image build` consumes the same `<image>:cache-<platform>` tag that `push-image-cache.yml` publishes from main. Fix the stale python tag in the fallback and refresh the comments that still claimed the job "will not use cache". (cherry picked from commit f3843d9) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Backport successfully created: v3-2-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
potiuk
added a commit
that referenced
this pull request
Apr 13, 2026
…ache (#65108) (#65144) The `build-docs` job in `publish-docs-to-s3.yml` was rebuilding the CI image from scratch because it never logged in to ghcr.io, so Buildx could not reach the registry cache tag pushed by the main "Test" workflow. The manual `docker buildx build` fallback also had no `--cache-from`, and a hardcoded `python3.9` tag that no longer matched the Python version used for the docs build. Add a `docker login ghcr.io` step, set `DOCKER_CACHE=registry` and `PYTHON_MAJOR_MINOR_VERSION` explicitly, and grant `packages: read` on the job so `breeze ci-image build` consumes the same `<image>:cache-<platform>` tag that `push-image-cache.yml` publishes from main. Fix the stale python tag in the fallback and refresh the comments that still claimed the job "will not use cache". (cherry picked from commit f3843d9) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Contributor
|
Nice! |
dandanseo123
pushed a commit
to dandanseo123/airflow
that referenced
this pull request
Apr 13, 2026
…#65108) The `build-docs` job in `publish-docs-to-s3.yml` was rebuilding the CI image from scratch because it never logged in to ghcr.io, so Buildx could not reach the registry cache tag pushed by the main "Test" workflow. The manual `docker buildx build` fallback also had no `--cache-from`, and a hardcoded `python3.9` tag that no longer matched the Python version used for the docs build. Add a `docker login ghcr.io` step, set `DOCKER_CACHE=registry` and `PYTHON_MAJOR_MINOR_VERSION` explicitly, and grant `packages: read` on the job so `breeze ci-image build` consumes the same `<image>:cache-<platform>` tag that `push-image-cache.yml` publishes from main. Fix the stale python tag in the fallback and refresh the comments that still claimed the job "will not use cache".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
build-docsjob inpublish-docs-to-s3.ymlwas rebuilding the CI image from scratch onevery run because it never logged in to
ghcr.io, so Buildx could not reach the registrycache tag (
<image>:cache-<platform>) that the main "Test" workflow pushes viapush-image-cache.yml. The manualdocker buildx buildfallback also had no--cache-fromand a hardcoded
python3.9tag that no longer matched the Python version used for the docsbuild.
Summary
docker login ghcr.iostep inbuild-docssobreeze ci-image buildcan actuallypull the registry buildx cache published by the last "main" Test workflow.
DOCKER_CACHE=registryandPYTHON_MAJOR_MINOR_VERSIONexplicitly on the job so thecache lookup is deterministic and matches
default-python-version.packages: readto thebuild-docsjob.python3.9tag in the buildx fallback so it derives fromPYTHON_MAJOR_MINOR_VERSION.the fallback path (no
--cache-from) still exists for old refs whosepyproject.tomlhasdrifted from main.
The fast path now hits the same registry cache the main Test workflow pushes; the fallback
is untouched and still handles the old-ref drift case.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.6) following the guidelines