Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge Gecko image_builder (with all history) into Taskgraph #507

Draft
wants to merge 74 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
38f3236
Bug 1226413 - Allow task images to be built upon push r=wcosta
Nov 16, 2015
f82285b
Bug 1266719 - Add retry and timeout options when downloading image co…
Apr 22, 2016
acbe39d
Bug 1275276 - Fail if image context cannot be downloaded r=jhford
May 24, 2016
7b35ba1
Bug 1302763 - Move docker images out of testing/docker into taskclust…
CuriousLearner Oct 20, 2016
5c08844
Backed out changeset d15798b73b9e (bug 1302763) for taskcluster build…
KWierso Oct 20, 2016
e1f52a2
Bug 1316183 - Compress docker images with zstd. r=dustin
jonasfj Nov 7, 2016
3f199c4
Bug 1318466 - Fix image_builder to fail if docker socket returns 404.…
jonasfj Nov 17, 2016
39bc99c
Bug 1319449 - Set -o pipefail and other robustness improvements for i…
jonasfj Nov 22, 2016
f2173ae
Bug 1302763 - Move docker images out of testing/docker into taskclust…
CuriousLearner Oct 20, 2016
b81397c
Backed out changeset fe7303de56be (bug 1302763) for failing webdriver…
Archaeopteryx Dec 19, 2016
e6e287a
Bug 1324414 - Reference prebuilt docker images by HASH. r=dustin
jonasfj Dec 19, 2016
cd26cab
Bug 1302763 - Move docker images out of testing/docker into taskclust…
CuriousLearner Oct 20, 2016
bd83c02
Bug 1325132 - Set correct docker image hashes; r=garndt
jonasfj Dec 21, 2016
7dd2858
Bug 1329701 - robustcheckout.py moved r=gps
tomrittervg Jan 9, 2017
7314c73
Bug 1350447 - Use python-zstandard for Docker image compression; r=du…
indygreg Mar 28, 2017
7f3074a
Bug 1350447 - Use --no-install-recommends in image_builder; r=dustin
indygreg Mar 28, 2017
880ebbb
Bug 1350437 - Bump image_builder version and hash; r=aki
indygreg Mar 29, 2017
881520c
Bug 1355731 - Move tooltool to mozbuild.action. r=gps
glandium Feb 22, 2017
6b851ca
Bug 1391476 - Require that all cache paths be declared as volumes; r=…
indygreg Aug 23, 2017
849089d
Bug 1391476 - Automatically set cache/volume permissions in run-task;…
indygreg Aug 23, 2017
cde0a97
Bug 1394883: use /builds/worker in image building; r=gps
djmitche Aug 30, 2017
4f6831d
Backed out 4 changesets (bug 1394883) for Windows signing chain-of-tr…
philor Sep 1, 2017
1f7aa83
Bug 1394883: use /builds/worker in image building; r=gps
djmitche Aug 30, 2017
ded8173
Bug 1425137: Get rid of references to quay.io in taskcluster; r=dustin
tomprince Dec 13, 2017
d78b62b
Bug 1426445: Add sanity check that worker uid/gid is 1000 in run-task…
tomprince Jan 2, 2018
8d2e90e
Backed out changeset 15a9e149f2db (bug 1426445) for build bustage
indygreg Jan 5, 2018
05da8d9
Bug 1429669 - Separate mercurial conf from install-mercurial.sh. r=du…
glandium Jan 11, 2018
d3b8f27
Partially backout changeset 22587e6ef7ca from bug 1429669 on a CLOSED…
glandium Jan 12, 2018
77f61a2
Bug 1430037 - Use the in-tree image builder image to build docker ima…
glandium Jan 11, 2018
1f764a0
Bug 1430037 - Use sparse checkouts for docker image builds. r=dustin
glandium Jan 11, 2018
b7b8410
Backed out 2 changesets (bug 1430037) for bustage on beta r=glandium
shindli Jan 17, 2018
687ca52
Bug 1430037 - Use the in-tree image builder image to build docker ima…
glandium Jan 11, 2018
4e0c9f0
Bug 1430037 - Use sparse checkouts for docker image builds. r=dustin
glandium Jan 11, 2018
8582d0a
Bug 1429996 - Revert partial backout of changeset 22587e6ef7ca from b…
glandium Jan 18, 2018
1cde08c
Bug 1432768 Linting compliance for image builder scripts r=Callek
Jan 24, 2018
909ed7a
Backed out changeset d0dc56c1e423 (bug 1432768) for bustage taskclust…
elizabal Jan 24, 2018
74cba9a
Bug 1432768 Linting compliance for image builder scripts r=Callek
Jan 24, 2018
60b9b0e
Bug 1432390 - Make `mach taskcluster-build-image` talk directly to th…
glandium Jan 25, 2018
6528026
Bug 1432390 - Allow to derive docker images from other in-tree images…
glandium Jan 25, 2018
d9166e0
Bug 1432627 Add shellcheck linter to CI r=ahal
Jan 29, 2018
5ec1b85
Backed out changeset 85794e045326 (bug 1432627) for shell lint failur…
elizabal Jan 29, 2018
9d54476
Bug 1432627 Add shellcheck linter to CI r=ahal
Jan 29, 2018
8575bec
Bug 1459737 - Move run-task into taskcluster/scripts; r=dustin
indygreg May 5, 2018
6ce0e80
Bug 1460475 - Install Python 3 on image_builder; r=dustin
indygreg May 10, 2018
2071da4
Bug 1460475 - Upgrade python-zstandard in image_builder; r=dustin
indygreg May 10, 2018
5b93e1c
Bug 1460475 - Port download-and-compress to Python 3; r=dustin
indygreg May 10, 2018
cb8abbe
Bug 1498640 - deploy latest image_builder image r=glandium
djmitche Oct 31, 2018
d1f5774
Backed out changeset f33b9e6aca06 (bug 1498640) for mda permafailures…
shindli Nov 1, 2018
02889a8
Bug 1498640 - deploy latest image_builder image r=glandium
djmitche Oct 31, 2018
90ec358
Backed out 5 changesets (bug 1498640, bug 1436037, bug 1497660, bug 1…
AndreeaPavel Nov 15, 2018
3e3adf1
Bug 1498640 - deploy latest image_builder image r=glandium
djmitche Oct 31, 2018
264f499
Bug 1512188 - Rename --vcs-checkout to --firefox-checkout; r=tomprince
indygreg Dec 11, 2018
3fb2433
Bug 1512188 - Rename --sparse-profile to --firefox-sparse-profile; r=…
indygreg Dec 11, 2018
999227f
Bug 1512188 - Revert to "gecko" for vcs naming; r=tomprince
indygreg Dec 11, 2018
da9bae3
Backed out 12 changesets (bug 1512285, bug 1512188) for toolchain fai…
dgluca Dec 11, 2018
62b3f4b
Bug 1513429 - Upgrade image_builder to Ubuntu 18.04; r=dustin
indygreg Dec 19, 2018
5e2d74b
Backed out changeset e52ecfe8a7b1 (bug 1513429) for toolchain failure…
bogdant-old Dec 19, 2018
a6e56ee
Bug 1513429 - Upgrade image_builder to Ubuntu 18.04; r=dustin
indygreg Dec 19, 2018
7b00d6c
Bug 1512188 - Rename --vcs-checkout to --gecko-checkout; r=tomprince,…
indygreg Dec 29, 2018
74cfb50
Bug 1512188 - Rename --sparse-profile to --gecko-sparse-profile; r=to…
indygreg Dec 29, 2018
6c0d397
Backed out 8 changesets (bug 1512285, bug 1512188) for fetch bustages…
bogdant-old Dec 29, 2018
767a089
Bug 1512188 - Rename --vcs-checkout to --gecko-checkout; r=tomprince,…
indygreg Dec 31, 2018
ef0519c
Bug 1512188 - Rename --sparse-profile to --gecko-sparse-profile; r=to…
indygreg Dec 31, 2018
79c7dbc
Bug 1527394: Squash docker images before exporting/compressing them r…
Feb 23, 2019
ed30ea6
Bug 1569355 - Upgrade python-zstandard to 0.11.1. r=tomprince
glandium Jul 25, 2019
16d778b
Bug 1626058: Build docker images with kaniko, instead of dind; r=glan…
tomprince Jul 7, 2020
ef30b99
Bug 1651227 - Use the in-tree kaniko image builder image to build doc…
glandium Jul 9, 2020
d7b0077
Bug 1617369 - Reformat recent rust changes with rustfmt r=emilio
sylvestre Jul 12, 2020
04026e1
Bug 1660964 - Upgrade skopeo to 1.1.1 and kaniko to 1.0.0. r=aki
glandium Aug 25, 2020
772b5cd
Backed out changeset a716e6202487 (bug 1660964) for Gecko Decision Ta…
Aug 26, 2020
7eac121
Bug 1660964 - Upgrade skopeo to 1.1.1 and kaniko to 1.0.0. r=aki
glandium Aug 26, 2020
aee764c
Bug 1662305 - Remove workaround for shortcomings from kaniko < 1. r=t…
glandium Sep 1, 2020
91ccdce
Bug 1829618 - Enable Prettier on json files (automatic fixes). r=perf…
Standard8 May 9, 2023
4227f27
ci: add 'docker-image' task for 'image-builder'
ahal May 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions taskcluster/docker/image-builder/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

FROM golang:1.14 as skopeo

WORKDIR /go/src/
RUN ["git", "clone", "--no-checkout", "--depth=1", "--branch=v1.1.1", "https://github.com/containers/skopeo", "."]
RUN ["git", "checkout", "67abbb3cefbdc876447583d5ea45e76bf441eba7"]
ENV GO111MODULE=on CGO_ENABLED=0
RUN ["go", "build", \
"-mod=vendor", "-o", "out/skopeo", \
"-tags", "exclude_graphdriver_devicemapper exclude_graphdriver_btrfs containers_image_openpgp", \
# Set unixTempDirForBigFiles so skopeo will extract in a directory hidden by kaniko
# We create the directory below.
"-ldflags", " -X github.com/containers/image/v5/internal/tmpdir.unixTempDirForBigFiles=/workspace/tmp -X github.com/containers/image/v5/signature.systemDefaultPolicyPath=/kaniko/containers/policy.json -extldflags \"-static\" -w -s", \
"./cmd/skopeo"]

FROM golang:1.14 as kaniko
WORKDIR /go/src/
RUN ["git", "clone", "--no-checkout", "--depth=1", "--branch=v1.0.0", "https://github.com/GoogleContainerTools/kaniko", "."]
RUN ["git", "checkout", "146ec6a9cd6f87b4a12e8119ded575d5edca35ac"]
RUN ["make"]

# Build the `build-image` command as a static binary using musl
# The setup is loosely based on a stripped down version of
# https://github.com/emk/rust-musl-builder/blob/master/Dockerfile
FROM debian:buster as build-image

COPY apt.conf /etc/apt/apt.conf.d/99taskcluster

RUN apt-get update && \
apt-get install \
build-essential \
ca-certificates \
curl \
musl-dev \
musl-tools \
&& \
useradd rust --user-group --create-home --shell /bin/bash

# Run all further code as user `rust`, and create our working directories
# as the appropriate user.
USER rust

# Set up our path with all our binary directories, including those for the
# musl-gcc toolchain and for our Rust toolchain.
ENV PATH=/home/rust/.cargo/bin:$PATH

# The Rust toolchain to use when building our image. Set by `hooks/build`.
ENV TOOLCHAIN=1.42.0 \
TARGET=x86_64-unknown-linux-musl

# Install our Rust toolchain and the `musl` target. We patch the
# command-line we pass to the installer so that it won't attempt to
# interact with the user or fool around with TTYs. We also set the default
# `--target` to musl so that our users don't need to keep overriding it
# manually.
RUN curl https://sh.rustup.rs -sSf | \
sh -s -- -y \
--profile minimal \
--default-toolchain $TOOLCHAIN \
--target $TARGET

# Expect our source code to live in /home/rust/src. We'll run the build as
# user `rust`, which will be uid 1000, gid 1000 outside the container.
RUN mkdir -p /home/rust/src
WORKDIR /home/rust/src
# Add our source code.
ADD --chown=rust:rust build-image/ ./

# --out-dir is not yet stable
ENV RUSTC_BOOTSTRAP=1
# Build our application.
RUN ["cargo", "build", "--target", "x86_64-unknown-linux-musl", "--out-dir=bin", "--release", "-Zunstable-options"]

FROM scratch as empty

FROM scratch

COPY --from=skopeo /go/src/out/skopeo /kaniko-bootstrap/skopeo
COPY --from=kaniko /go/src/out/executor /kaniko-bootstrap/executor
COPY --from=build-image \
/home/rust/src/bin/build-image \
/kaniko-bootstrap/build-image

ADD https://mkcert.org/generate/ /kaniko-bootstrap/ssl/certs/ca-certificats.crt
ENV SSL_CERT_DIR=/kaniko/ssl/certs

ADD policy.json /kaniko-bootstrap/containers/policy.json

ENV HOME /root
ENV USER /root
WORKDIR /workspace

ENV PATH /usr/local/bin:/kaniko

VOLUME /workspace
ENTRYPOINT ["/kaniko-bootstrap/build-image"]
1 change: 1 addition & 0 deletions taskcluster/docker/image-builder/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
4.0.0
5 changes: 5 additions & 0 deletions taskcluster/docker/image-builder/apt.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
quiet "true";
APT::Get::Assume-Yes "true";
APT::Install-Recommends "false";
Acquire::Check-Valid-Until "false";
Acquire::Retries "5";
Loading
Loading