Skip to content

feat(docker): add --multi-arch flag for cross-platform image builds#18522

Merged
yihua merged 1 commit into
apache:masterfrom
yihua:feat-multi-arch-docker
Apr 17, 2026
Merged

feat(docker): add --multi-arch flag for cross-platform image builds#18522
yihua merged 1 commit into
apache:masterfrom
yihua:feat-multi-arch-docker

Conversation

@yihua
Copy link
Copy Markdown
Contributor

@yihua yihua commented Apr 17, 2026

Describe the issue this Pull Request addresses

The current Docker build script only supports one architecture based on the local hardware.

Closes: #18517

Summary and Changelog

  • Add --multi-arch flag to build_docker_images.sh for building both linux/amd64 and linux/arm64 images via docker buildx
  • Without the flag, behavior is unchanged (builds for local architecture only)
  • Multi-arch builds push directly to the registry since buildx cannot load multi-platform images into the local daemon

Usage:

# Local build (unchanged)
./build_docker_images.sh --hadoop-version 3.4.0 --spark-version 4.0.1 --hive-version 3.1.3

# Multi-arch build + push
./build_docker_images.sh --hadoop-version 3.4.0 --spark-version 4.0.1 --hive-version 3.1.3 --multi-arch

Requires a multi-platform buildx builder and docker login:

docker buildx create --name multiarch --driver docker-container --use

Impact

Makes test docker image generation easier from Macbook

Risk Level

none

Documentation Update

Usage docs updated in the script

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

Copy link
Copy Markdown
Contributor Author

@yihua yihua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

No reviewable code files in this PR.

@github-actions github-actions Bot added the size:S PR with lines of changes in (10, 100] label Apr 17, 2026
@hudi-bot
Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Copy link
Copy Markdown
Member

@voonhous voonhous left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@yihua yihua merged commit 97f9628 into apache:master Apr 17, 2026
57 of 60 checks passed
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.87%. Comparing base (a369773) to head (67060a5).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18522      +/-   ##
============================================
+ Coverage     68.84%   68.87%   +0.03%     
- Complexity    28250    28287      +37     
============================================
  Files          2464     2464              
  Lines        135442   135568     +126     
  Branches      16417    16443      +26     
============================================
+ Hits          93239    93372     +133     
+ Misses        34821    34806      -15     
- Partials       7382     7390       +8     
Flag Coverage Δ
common-and-other-modules 44.64% <ø> (+0.06%) ⬆️
hadoop-mr-java-client 44.75% <ø> (-0.06%) ⬇️
spark-client-hadoop-common 48.40% <ø> (-0.05%) ⬇️
spark-java-tests 48.91% <ø> (-0.02%) ⬇️
spark-scala-tests 45.43% <ø> (-0.04%) ⬇️
utilities 38.19% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 13 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S PR with lines of changes in (10, 100]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parameterise docker-compose ARM64/AMD64 builds

4 participants