Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build and push Docker image based on Chainguard base image #13137

Merged
merged 15 commits into from
May 24, 2024
Merged

build and push Docker image based on Chainguard base image #13137

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
7d0feee
support wolfi
v1v May 10, 2024
a89834b
Use one Dockerfile with args to configure wolfi
v1v May 10, 2024
369c5be
tag docker images
v1v May 10, 2024
a1ea9f8
ci: push docker images for no forked PRs
v1v May 10, 2024
b7527f1
avoid failing for the time being just in case parallel build scause a…
v1v May 10, 2024
de52938
login beforehand
v1v May 10, 2024
c30bbfd
ci: enable wolfi docker image generation with GENERATE_WOLFI_IMAGES
v1v May 10, 2024
569a5b9
simplify logic ci
v1v May 10, 2024
de225d7
set env and rename jobs
v1v May 10, 2024
1bb262f
fix
v1v May 10, 2024
8509876
use placeholder for the chainguard
v1v May 15, 2024
e403623
fix duplicated
v1v May 15, 2024
f1fe3f9
Update .buildkite/package.yml
v1v May 23, 2024
0db7be1
Update .buildkite/package.yml
v1v May 23, 2024
88fcc8b
Merge branch 'main' into feature/docker-images-wolfi
v1v May 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .buildkite/package.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ notify:
env:
IMAGE_UBUNTU_X86_64: "family/core-ubuntu-2004"
IMAGE_UBUNTU_ARM_64: "core-ubuntu-2004-aarch64"
GENERATE_WOLFI_IMAGES: "true"

steps:
- group: "Package"
Expand Down
25 changes: 25 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,28 @@ jobs:
go-version-file: go.mod
cache: false
- run: make package-snapshot
env:
GH_TOKEN: ${{ github.token }}

test-package-and-push:
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false)
env:
GENERATE_WOLFI_IMAGES: true
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: false
- uses: elastic/apm-pipeline-library/.github/actions/docker-login@current
with:
registry: docker.elastic.co
secret: secret/observability-team/ci/docker-registry/prod
url: ${{ secrets.VAULT_ADDR }}
roleId: ${{ secrets.VAULT_ROLE_ID }}
secretId: ${{ secrets.VAULT_SECRET_ID }}
- run: make package-snapshot
env:
GH_TOKEN: ${{ github.token }}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove some warnings in the github logs, regarding missing variable when running gh in the GitHub runner

- run: make publish-docker-images
31 changes: 28 additions & 3 deletions packaging.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,38 @@ DOCKER_IMAGES := \
build/docker/apm-server-ubi-$(APM_SERVER_VERSION).txt \
build/docker/apm-server-ubi-$(APM_SERVER_VERSION)-SNAPSHOT.txt

# If GENERATE_WOLFI_IMAGES is set then generate wolfi docker images.
ifdef GENERATE_WOLFI_IMAGES
DOCKER_IMAGES := $(DOCKER_IMAGES) \
build/docker/apm-server-wolfi-$(APM_SERVER_VERSION).txt \
build/docker/apm-server-wolfi-$(APM_SERVER_VERSION)-SNAPSHOT.txt
endif

build/docker/%.txt: DOCKER_IMAGE_TAG := docker.elastic.co/apm/apm-server:%
build/docker/%.txt: VERSION := $(APM_SERVER_VERSION)
build/docker/%-SNAPSHOT.txt: VERSION := $(APM_SERVER_VERSION)-SNAPSHOT
build/docker/apm-server-ubi-%.txt: DOCKER_BUILD_ARGS+=--build-arg BASE_IMAGE=docker.elastic.co/ubi9/ubi-minimal
build/docker/apm-server-wolfi-%.txt: DOCKER_BUILD_ARGS+=--build-arg BASE_IMAGE=docker.elastic.co/wolfi/chainguard-base:20230214 --build-arg BASE_GOLANG_IMAGE=docker.elastic.co/wolfi/go
v1v marked this conversation as resolved.
Show resolved Hide resolved

INTERNAL_DOCKER_IMAGE := docker.elastic.co/observability-ci/apm-server-internal
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should help create docker images and publish them to our internal docker registry - this is handy for validating changes before anything related to the DRA and in GitHub actions.

We are safe since these docker images won't be public available for the time being:

$ docker pull docker.elastic.co/observability-ci/apm-server-internal:8.15.0-SNAPSHOT-wolfi
Error response from daemon: pull access denied for docker.elastic.co/observability-ci/apm-server-internal, repository does not exist or may require 'docker login': denied: requested access to the resource is denied


.PHONY: $(DOCKER_IMAGES)
$(DOCKER_IMAGES):
@mkdir -p $(@D)
docker build --iidfile="$(@)" --build-arg GOLANG_VERSION=$(GOLANG_VERSION) --build-arg VERSION=$(VERSION) $(DOCKER_BUILD_ARGS) -f packaging/docker/Dockerfile .

# Docker image tarballs. We distribute UBI8 Docker images only for AMD64.
docker build --iidfile="$(@)" \
--build-arg GOLANG_VERSION=$(GOLANG_VERSION) \
--build-arg VERSION=$(VERSION) \
$(DOCKER_BUILD_ARGS) \
--tag $(INTERNAL_DOCKER_IMAGE):$(VERSION)$(if $(findstring arm64,$(GOARCH)),-arm64)$(if $(findstring wolfi,$(@)),-wolfi) \
-f packaging/docker/Dockerfile .

# Docker image tarballs. We distribute UBI Docker images only for AMD64.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs: no ubi8 but ubi, #11977

DOCKER_IMAGE_SUFFIX := docker-image$(if $(findstring arm64,$(GOARCH)),-arm64).tar.gz
DOCKER_IMAGE_PREFIXES := apm-server $(if $(findstring amd64,$(GOARCH)), apm-server-ubi)
# If GENERATE_WOLFI_IMAGES is set then generate wolfi docker images.
ifdef GENERATE_WOLFI_IMAGES
DOCKER_IMAGE_PREFIXES := $(DOCKER_IMAGE_PREFIXES) apm-server-wolfi
endif
DOCKER_IMAGE_RELEASE_TARBALLS := $(patsubst %, $(DISTDIR)/%-$(APM_SERVER_VERSION)-$(DOCKER_IMAGE_SUFFIX), $(DOCKER_IMAGE_PREFIXES))
DOCKER_IMAGE_SNAPSHOT_TARBALLS := $(patsubst %, $(DISTDIR)/%-$(APM_SERVER_VERSION)-SNAPSHOT-$(DOCKER_IMAGE_SUFFIX), $(DOCKER_IMAGE_PREFIXES))

Expand Down Expand Up @@ -186,7 +205,10 @@ build/dependencies-$(APM_SERVER_VERSION)-SNAPSHOT.csv: build/dependencies-$(APM_
cp $< $@

package-docker: $(DOCKER_IMAGE_RELEASE_TARBALLS)
@echo ">> $(DOCKER_IMAGE_RELEASE_TARBALLS)"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to help with printing what it did


package-docker-snapshot: $(DOCKER_IMAGE_SNAPSHOT_TARBALLS)
@echo ">> $(DOCKER_IMAGE_SNAPSHOT_TARBALLS)"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to help with printing what it did


package: \
package-docker \
Expand All @@ -200,3 +222,6 @@ package-snapshot: \
$(DOCKER_IMAGE_SNAPSHOT_TARBALLS) \
$(DISTDIR)/apm-server-ironbank-$(APM_SERVER_VERSION)-SNAPSHOT-docker-build-context.tar.gz \
build/dependencies-$(APM_SERVER_VERSION)-SNAPSHOT.csv

publish-docker-images:
docker push --all-tags $(INTERNAL_DOCKER_IMAGE)
14 changes: 11 additions & 3 deletions packaging/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,32 @@
ARG BASE_IMAGE=ubuntu:22.04
ARG BASE_GOLANG_IMAGE=golang
ARG GOLANG_VERSION

################################################################################
# Build stage 0
# Build the apm-server binary. The golang image version is kept
# up to date with go.mod by Makefile.
FROM golang:${GOLANG_VERSION} as builder
################################################################################
FROM ${BASE_GOLANG_IMAGE}:${GOLANG_VERSION} as builder
WORKDIR /src
COPY go.mod go.sum /src/
COPY go.mod go.sum .go-version /src/
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes a warning when building the docker image

COPY internal/glog/go.mod /src/internal/glog/go.mod
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY Makefile *.mk /src
COPY cmd /src/cmd
COPY internal /src/internal
COPY x-pack /src/x-pack
COPY .git /src/.git
COPY script /src/script
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes a warning when building the docker image


ENV GOROOT /usr/local/go
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrequired for building the wolfi version and in fact it's not needed for the traditional docker image

RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
make apm-server

################################################################################
# Build stage 1
# Copy prepared files from the previous stage and complete the image.
################################################################################
FROM ${BASE_IMAGE}
ARG TARGETARCH
ARG BUILD_DATE
Expand Down