Skip to content

fix(ci): stop py311/py312 docker builds from silently matching lean - #42509

Open
rusackas wants to merge 1 commit into
masterfrom
fix/docker-matrix-py311-py312-pyver-override
Open

fix(ci): stop py311/py312 docker builds from silently matching lean#42509
rusackas wants to merge 1 commit into
masterfrom
fix/docker-matrix-py311-py312-pyver-override

Conversation

@rusackas

Copy link
Copy Markdown
Member

SUMMARY

While looking into whether docker/bake-action would help the docker build matrix (dev/lean/websocket/dockerize/py311/py312), I noticed something unrelated to bake: docker.yml appends --build-arg PY_VER=3.11.14-slim-trixie via --extra-flags for every preset in the matrix, so it can pin dev/lean to the Dockerfile's actual supported Python version (supersetbot's own default for those two is stale). But docker/buildx keeps the last value for a repeated --build-arg key, and that override was landing after supersetbot's own --build-arg PY_VER for py311/py312 too, silently clobbering the one thing that's supposed to make those two presets different from lean. lean, py311, and py312 have been building the same image.

This excludes py311/py312 from that override so their own PY_VER pin actually survives.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A, CI-only change.

TESTING INSTRUCTIONS

zizmor and pre-commit both pass clean. The real test is the docker-build matrix on this PR itself: py311 and py312 should now show a different base= label than lean in the "Print docker stats" step (docker history $IMAGE_TAG), where before all three matched.

ADDITIONAL INFORMATION

While in here, a few other things came up that I'm treating as separate follow-ups rather than folding into this PR:

  • docker/bake-action itself: dev/lean/py311/py312 all share one Dockerfile and its ancestor stages (notably the Node/frontend build stage, which doesn't even depend on PY_VER), so building them via one docker buildx bake invocation instead of 4 separate matrix legs would cut real redundant work. websocket and dockerize are unrelated Dockerfiles with no shared FROM ancestry, so they wouldn't gain anything from being folded in. This is a bigger lift: it means teaching supersetbot to emit a bake file, and it trades runner-level parallelism (today's legs run concurrently on separate runners) for one job doing the shared work once, so it's a compute/cost win more than a guaranteed wall-clock win unless paired with a beefier runner. I'll open that as its own PR against supersetbot.

  • There's already a registry-backed buildx cache (--cache-from/--cache-to against apache/superset-cache:<pyVer>) per preset, so this isn't a from-scratch-every-time situation, just one where shared stages get pulled/pushed redundantly across legs instead of built once.

  • Unrelated oddity I didn't touch: the build step references $EVENT, $RELEASE, and $FORCE_LATEST, none of which are set anywhere in this workflow. supersetbot falls back to deriving context/ref from GITHUB_EVENT_NAME/GITHUB_REF when they're empty, so this looks harmless as-is, but the dead variables are worth a cleanup pass sometime.

  • Has associated issue:

  • Required feature flags:

  • Changes UI

  • Includes DB Migration

  • Introduces new feature or API

  • Removes existing feature or API

docker.yml's build step appends --build-arg PY_VER=3.11.14-slim-trixie
via --extra-flags for every matrix leg, so it can pin dev/lean to the
Dockerfile's supported Python version even though supersetbot's own
default for those two presets is stale. But docker/buildx keeps the
last value for a repeated --build-arg key, and that override was
landing after supersetbot's own --build-arg PY_VER for the py311/py312
presets too, clobbering the one thing that's supposed to make those
two presets different from lean. All three have been building the
same image.

Exclude py311/py312 from the override so their own PY_VER pin
survives.
@dosubot dosubot Bot added the github_actions Pull requests that update GitHub Actions code label Jul 28, 2026
@bito-code-review

bito-code-review Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Bito Automatic Review Skipped - Files Excluded

Bito didn't auto-review this change because all changed files are in the exclusion list for automatic reviews. No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the excluded files settings here, or contact your Bito workspace admin at evan@preset.io.

@rusackas
rusackas requested a review from Copilot July 28, 2026 17:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Fixes the CI docker build matrix so the py311/py312 presets no longer get their PY_VER build arg unintentionally overridden (making them silently build the same image as lean).

Changes:

  • Builds EXTRA_FLAGS dynamically to conditionally append the PY_VER=3.11.14-slim-trixie override only for non-py311/py312 presets
  • Adds inline documentation explaining the repeated --build-arg PY_VER “last value wins” behavior and why the exclusion is needed

Comment thread .github/workflows/docker.yml
@bito-code-review

Copy link
Copy Markdown
Contributor

The suggestion to use an allowlist instead of an exclusion list is appropriate. It improves maintainability by ensuring that new presets default to the standard behavior, preventing accidental silent overrides. You can implement this by checking if the preset is in the intended list of presets that require the override.

.github/workflows/docker.yml

EXTRA_FLAGS="--build-arg INCLUDE_CHROMIUM=false --tag $IMAGE_TAG"
          case "$BUILD_PRESET" in
            dev|lean)
              EXTRA_FLAGS="--build-arg PY_VER=3.11.14-slim-trixie $EXTRA_FLAGS"
              ;;
          esac

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code size/S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants