Skip to content

Split Dockerfile into parallel build groups for GitHub Actions #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hsbt merged 8 commits into from
Mar 4, 2026
Merged

Split Dockerfile into parallel build groups for GitHub Actions #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
31dfc6c
Split Dockerfile into parallel build groups for GitHub Actions
hsbt Mar 4, 2026
a23615f
Rename Docker build stages and GH matrix targets
hsbt Mar 4, 2026
1e1debd
Initial plan
Copilot Mar 4, 2026
663c99d
Initial plan
Copilot Mar 4, 2026
ffe5738
Fix missing Check-Valid-Until=false on second apt-get update in build…
Copilot Mar 4, 2026
b98af80
Merge pull request #24 from ruby/copilot/sub-pr-23
hsbt Mar 4, 2026
a974825
Move rdfind dedup to intermediate aggregator stage, remove from final…
Copilot Mar 4, 2026
168e434
Merge pull request #25 from ruby/copilot/sub-pr-23-again
hsbt Mar 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: Build Docker image

on:
push:
branches: [master]
workflow_dispatch:

env:
REGISTRY: docker.io
IMAGE_NAME: ${{ github.repository }}

permissions:
contents: read

jobs:
build-group:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
target:
- ruby-0.x-2.0
- ruby-1.2-1.9
- ruby-2.1-2.4
- ruby-2.5-2.7
- ruby-3.0-3.2
- ruby-3.3-4.0
steps:
- uses: actions/checkout@v4

- uses: docker/setup-buildx-action@v3

- uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}

- name: Build ${{ matrix.target }}
uses: docker/build-push-action@v6
with:
context: .
target: ${{ matrix.target }}
cache-from: |
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache-${{ matrix.target }}
cache-to: |
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache-${{ matrix.target }},mode=max

build-final:
needs: build-group
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: docker/setup-buildx-action@v3

- uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}

- name: Build and push final image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
cache-from: |
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache-ruby-0.x-2.0
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache-ruby-1.2-1.9
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache-ruby-2.1-2.4
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache-ruby-2.5-2.7
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache-ruby-3.0-3.2
type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:cache-ruby-3.3-4.0
150 changes: 104 additions & 46 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,14 @@ ARG variant=-slim
ARG mirror=http://deb.debian.org/debian
ARG system_ruby=ruby2.7

# Build for 0.*, 1.0*, 1.1*, 1.8 and 1.8.5
FROM debian:buster-slim
# rake -j interpret non-numeric argument as number of CPUs plus 3.
ARG j=numcpu_plus_alpha

# =============================================================================
# Base build environment: Debian Buster (for legacy Ruby versions)
# =============================================================================
FROM debian:buster-slim AS builder-buster
ENV DEBIAN_FRONTEND=noninteractive
ARG mirror

RUN echo "deb http://archive.debian.org/debian/ buster main contrib non-free" > /etc/apt/sources.list && \
echo "deb http://archive.debian.org/debian-security/ buster/updates main contrib non-free" >> /etc/apt/sources.list && \
Comment on lines 16 to 17
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stage uses archive.debian.org (Buster) and disables Acquire::Check-Valid-Until for the apt-get update here, but there is another apt-get update later in the same stage (after adding i386 + deb-src) that will still hit expired Valid-Until unless the check is also disabled/persisted. To avoid flaky/failing builds, apply Check-Valid-Until=false to the later update as well (or add an apt conf entry so it applies to all updates in this stage).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

@hsbt hsbt Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot open a new pull request to apply changes based on this feedback

Expand All @@ -17,7 +21,7 @@ RUN dpkg --add-architecture i386 \
&& echo "deb-src http://archive.debian.org/debian/ buster main" > /etc/apt/sources.list.d/deb-src.list \
&& echo 'Dpkg::Use-Pty "0";\nquiet "2";\nAPT::Install-Recommends "0";' > /etc/apt/apt.conf.d/99autopilot \
&& echo 'Acquire::HTTP::No-Cache "True";' > /etc/apt/apt.conf.d/99no-cache \
&& apt-get update \
&& apt-get update -o Acquire::Check-Valid-Until=false \
&& apt-get install \
build-essential \
gcc-multilib \
Expand All @@ -36,20 +40,25 @@ COPY lib/ruby_version.rb /all-ruby/lib/
COPY patch /all-ruby/patch/
RUN rake setup_build

# rake -j interpret non-numeric argument as number of CPUs plus 3.
# =============================================================================
# Ruby 0.x, 1.0, 1.1, 1.8.0-1.8.5, 2.0.0 on Debian Buster
# =============================================================================
FROM builder-buster AS ruby-0.x-2.0
ARG j=numcpu_plus_alpha

COPY versions/0.* versions/1.* versions/2.0.0* /all-ruby/versions/
COPY versions/0.* versions/1.0* versions/1.1* versions/1.8.0* versions/1.8.1* versions/1.8.2* versions/1.8.3* versions/1.8.4* versions/1.8.5* versions/2.0.0* /all-ruby/versions/
RUN rake -j ${j} all-0 all-1.0 all-1.1a all-1.1b all-1.1c all-1.1d all-1.8 all-1.8.5
RUN rake -j ${j} all-2.0.0

RUN rm -rf Rakefile versions/ patch/
RUN rm -rf DIST build/*/log build/*/ruby*/
RUN rm -rf build/*/man build/*/share/man build/*/share/doc build/*/share/ri
RUN rm -f build/*/lib/libruby-static.a
RUN rm -f build/*/bin/gcc build/*/bin/cc
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The find … -print0 | xargs -0 strip pipeline can fail when no files match the file | grep -q "not stripped" filter (GNU xargs will still invoke strip with no args). Consider avoiding xargs here (e.g., use find … -exec strip … {} +) or add --no-run-if-empty/-r to xargs; also quoting the filename in the file call would be safer.

Copilot uses AI. Check for mistakes.

FROM ${os}:${version}${variant}
# =============================================================================
# Base build environment: Debian Bullseye (for modern Ruby versions)
# =============================================================================
FROM ${os}:${version}${variant} AS builder-bullseye
ENV DEBIAN_FRONTEND=noninteractive
ARG mirror
ARG version
Expand All @@ -71,62 +80,103 @@ RUN dpkg --add-architecture i386 \
&& apt-get build-dep ${system_ruby} \
&& rm -rf /var/lib/apt/lists/*

COPY --from=0 /build-all-ruby/ /build-all-ruby
COPY --from=0 /all-ruby/ /all-ruby

WORKDIR /all-ruby

COPY Rakefile /all-ruby/
COPY lib/ruby_version.rb /all-ruby/lib/
COPY patch /all-ruby/patch/
RUN rake setup_build

# rake -j interpret non-numeric argument as number of CPUs plus 3.
# =============================================================================
# Ruby 1.2-1.8.7, 1.9.x
# =============================================================================
FROM builder-bullseye AS ruby-1.2-1.9
ARG j=numcpu_plus_alpha

COPY versions/1.* versions/2.1* versions/2.2* versions/2.3* versions/2.4* versions/2.5* versions/2.6* versions/2.7* versions/3.0* /all-ruby/versions/
COPY versions/1.2* versions/1.3* versions/1.4* versions/1.6* versions/1.8.6* versions/1.8.7* versions/1.9* /all-ruby/versions/
RUN rake -j ${j} all-1.2 all-1.3 all-1.4 all-1.6 all-1.8.6 all-1.8.7
RUN rake -j ${j} all-1.9.0 all-1.9.1 all-1.9.2
RUN rake -j ${j} all-1.9.3
RUN rake -j ${j} all-2.1
RUN rake -j ${j} all-2.2
RUN rake -j ${j} all-2.3
RUN rake -j ${j} all-2.4
RUN rake -j ${j} all-2.5
RUN rake -j ${j} all-2.6
RUN rake -j ${j} all-2.7
RUN rake -j ${j} all-3.0

COPY versions/3.1* /all-ruby/versions/
RUN rake -j ${j} all-3.1
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: this find … | xargs strip can invoke strip with no args if the file | grep filter selects nothing. Prefer find … -exec strip … {} + or add xargs --no-run-if-empty/-r (and quote the filename passed to file).

Copilot uses AI. Check for mistakes.

COPY versions/3.2* /all-ruby/versions/
RUN rake -j ${j} all-3.2
# =============================================================================
# Ruby 2.1-2.4
# =============================================================================
FROM builder-bullseye AS ruby-2.1-2.4
ARG j=numcpu_plus_alpha

COPY versions/3.3* /all-ruby/versions/
RUN rake -j ${j} all-3.3
COPY versions/2.1* versions/2.2* versions/2.3* versions/2.4* /all-ruby/versions/
RUN rake -j ${j} all-2.1 all-2.2 all-2.3 all-2.4

COPY versions/3.4* /all-ruby/versions/
RUN rake -j ${j} all-3.4
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: if the file | grep -q "not stripped" test matches nothing, xargs may still run strip with no arguments and fail the build. Use find … -exec strip … {} + or add xargs --no-run-if-empty/-r.

Copilot uses AI. Check for mistakes.

COPY versions/3.5* /all-ruby/versions/
RUN rake -j ${j} 3.5.0-preview1
# =============================================================================
# Ruby 2.5-2.7
# =============================================================================
FROM builder-bullseye AS ruby-2.5-2.7
ARG j=numcpu_plus_alpha

COPY versions/4.0* /all-ruby/versions/
RUN rake -j ${j} all-4.0
COPY versions/2.5* versions/2.6* versions/2.7* /all-ruby/versions/
RUN rake -j ${j} all-2.5 all-2.6 all-2.7

COPY lib/* /all-ruby/lib/
COPY all-ruby /all-ruby/
RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: xargs strip may be executed with no operands when the preceding find prints nothing, causing the layer to fail. Prefer find … -exec strip … {} + or add xargs --no-run-if-empty/-r.

Copilot uses AI. Check for mistakes.

# =============================================================================
# Ruby 3.0-3.2
# =============================================================================
FROM builder-bullseye AS ruby-3.0-3.2
ARG j=numcpu_plus_alpha

RUN rm -rf Rakefile versions/ patch/
RUN rm -rf DIST build/*/log build/*/ruby*/
RUN rm -rf build/*/man build/*/share/man build/*/share/doc build/*/share/ri
RUN rm -f build/*/lib/libruby-static.a
RUN rm -f build/*/bin/gcc build/*/bin/cc
COPY versions/3.0* versions/3.1* versions/3.2* /all-ruby/versions/
RUN rake -j ${j} all-3.0 all-3.1 all-3.2

RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: consider avoiding … -print0 | xargs -0 strip because xargs can run strip with no args if no files are selected. Use find … -exec strip … {} + or add xargs --no-run-if-empty/-r.

Copilot uses AI. Check for mistakes.

# =============================================================================
# Ruby 3.3-4.0
# =============================================================================
FROM builder-bullseye AS ruby-3.3-4.0
ARG j=numcpu_plus_alpha

COPY versions/3.3* versions/3.4* versions/3.5* versions/4.0* /all-ruby/versions/
RUN rake -j ${j} all-3.3 all-3.4 3.5.0-preview1 all-4.0

RUN rm -rf Rakefile versions/ patch/ DIST build/*/log build/*/ruby*/ \
build/*/man build/*/share/man build/*/share/doc build/*/share/ri && \
rm -f build/*/lib/libruby-static.a build/*/bin/gcc build/*/bin/cc
RUN find /build-all-ruby -type f \( -name ruby -o -name '*.so' \) -exec sh -c 'file $1 | grep -q "not stripped"' - '{}' \; -print0 | xargs -0 strip
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as other groups: xargs strip can fail if the file | grep predicate filters everything out (xargs may still invoke strip with no inputs). Prefer find … -exec strip … {} + or add xargs --no-run-if-empty/-r.

Copilot uses AI. Check for mistakes.

# =============================================================================
# Aggregator: Combine all build outputs and deduplicate with rdfind
# (rdfind is already installed in builder-bullseye)
# =============================================================================
FROM builder-bullseye AS aggregator

COPY --from=ruby-0.x-2.0 /build-all-ruby/ /build-all-ruby/
COPY --from=ruby-1.2-1.9 /build-all-ruby/ /build-all-ruby/
COPY --from=ruby-2.1-2.4 /build-all-ruby/ /build-all-ruby/
COPY --from=ruby-2.5-2.7 /build-all-ruby/ /build-all-ruby/
COPY --from=ruby-3.0-3.2 /build-all-ruby/ /build-all-ruby/
COPY --from=ruby-3.3-4.0 /build-all-ruby/ /build-all-ruby/

RUN rdfind -makehardlinks true -makeresultsfile false /build-all-ruby

# =============================================================================
# Final: Runtime image
# =============================================================================
FROM ${os}:${version}${variant}
ENV DEBIAN_FRONTEND=noninteractive
ARG mirror
Expand Down Expand Up @@ -159,7 +209,15 @@ RUN dpkg --add-architecture i386 \
${system_ruby} \
&& rm -rf /var/lib/apt/lists/*

COPY --from=1 /build-all-ruby/ /build-all-ruby
COPY --from=1 /all-ruby/ /all-ruby
COPY --from=aggregator /build-all-ruby/ /build-all-ruby/
COPY --from=ruby-0.x-2.0 /all-ruby/ /all-ruby/
COPY --from=ruby-1.2-1.9 /all-ruby/bin/ /all-ruby/bin/
COPY --from=ruby-2.1-2.4 /all-ruby/bin/ /all-ruby/bin/
COPY --from=ruby-2.5-2.7 /all-ruby/bin/ /all-ruby/bin/
COPY --from=ruby-3.0-3.2 /all-ruby/bin/ /all-ruby/bin/
COPY --from=ruby-3.3-4.0 /all-ruby/bin/ /all-ruby/bin/

COPY lib/* /all-ruby/lib/
COPY all-ruby /all-ruby/

WORKDIR /all-ruby