Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker image tool: implement local image builds #8141

Merged
merged 2 commits into from
Feb 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 10 additions & 0 deletions helm/nessie/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,16 @@ From Nessie repo root:
$ helm install --namespace nessie-ns nessie helm/nessie
```

#### Local Helm chart using a locally built Nessie image

Running minikube with a locally built image is a bit more complex:
1. Add the `registry` addon to `minikube` using `minikube addons enable registry`
2. Allow using an insecure registry using `minikube config set insecure-registry "$(minikube ip)"`
3. Build the image using `tools/dockerbuild/build-push-images.sh -g :nessie-quarkus -p servers/quarkus-server projectnessie/nessie-local -l -t podman`
4. Push the image using `podman push --tls-verify=false localhost/projectnessie/nessie-local:latest docker://$(minikube ip):5000/projectnessie/nessie-local:latest`
5. Use the following Helm values: `image.repository=192.168.49.2:5000/projectnessie/nessie-local:latest` and `image.pullPolicy=Always`
6. Install the Helm chart as described above

### Uninstalling the chart

```bash
Expand Down
20 changes: 15 additions & 5 deletions tools/dockerbuild/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ The `build-push-images.sh` script is meant to build Java and optionally native i
restricted to (or specialized for) `:nessie-quarkus`, but can be used with basically any Quarkus
application.

Minimal options (use `./build-push-images.sh -h` for an up-to-date listing) are:
Minimal options (use `./build-push-images.sh --help` for an up-to-date listing) are:

* `-g <gradle-project-name>` in the `:my-project-name` syntax
* `-p <gradle-project-directory>` like `servers/quarkus-server`
* `--gradle-project <gradle-project-name>` in the `:my-project-name` syntax
* `--project-dir <gradle-project-directory>` like `servers/quarkus-server`
* the image name as an argument, for example `projectnessie/nessie`

The image name also defines where the built images are pushed to. By (Docker) default, it's Docker
Expand All @@ -20,13 +20,23 @@ example (see local Docker registry notes below):

```bash
tools/dockerbuild/build-push-images.sh \
-g :nessie-quarkus \
-p servers/quarkus-server \
--gradle-project :nessie-quarkus \
--project-dir servers/quarkus-server \
localhost:5000/projectnessie/nessie-local
```

Hint: The images are pushed to the local registry, those will **not** show up in `docker images`.

## Build an image for local use

```bash
tools/dockerbuild/build-push-images.sh \
--gradle-project :nessie-quarkus \
--project-dir servers/quarkus-server \
--local \
localhost/projectnessie/nessie-local
```

## Updating your local Docker registry

There's a `docker-registry` Debian/Ubuntu package. The registry configuration might need some tweaks
Expand Down
107 changes: 67 additions & 40 deletions tools/dockerbuild/build-push-images.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,14 @@ set -e

IMAGE_NAME=""
GITHUB=0
LOCAL=0
ARTIFACTS=""
GRADLE_PROJECT=""
PROJECT_DIR=""
DOCKERFILE="Dockerfile-server"

TOOL="$(which docker > /dev/null && echo docker || echo podman)"

if [[ -n ${GITHUB_ENV} ]]; then
GITHUB=1
fi
Expand All @@ -45,12 +48,18 @@ function usage() {
-d | --dockerfile <file> Dockerfile to use (default: Dockerfile-server)
-gh | --github GitHub actions mode
-a | --artifacts-dir <dir> Directory to place uber-jars in
-l | --local Only build the image for local use (not multi-platform),
not pushed to a registry. Can build with 'docker' and 'podman'.
-t | --tool Name of the podman/docker/podman-remote tool to use.

Note: multiplatform builds only work with docker buildx, not implemented for podman.

GitHub mode is automatically enabled, when GITHUB_ENV is present. -a is mandatory in GitHub mode.

Examples:
$0 -d Dockerfile-server -g :nessie-quarkus -p servers/quarkus-server nessie-unstable
$0 -d Dockerfile-gctool -g :nessie-gc-tool -p gc/gc-tool nessie-gc-unstable
$0 --local --dockerfile Dockerfile-server --gradle-project :nessie-quarkus --project-dir servers/quarkus-server nessie-local
$0 --dockerfile Dockerfile-server --gradle-project :nessie-quarkus --project-dir servers/quarkus-server nessie-unstable
$0 --dockerfile Dockerfile-gctool --gradle-project :nessie-gc-tool --project-dir gc/gc-tool nessie-gc-unstable
!
}

Expand Down Expand Up @@ -88,6 +97,13 @@ while [[ $# -gt 0 ]]; do
-gh | --github)
GITHUB=1
;;
-l | --local)
LOCAL=1
;;
-t | --TOOL)
TOOL="$2"
shift
;;
-h | --help)
usage
exit 0
Expand Down Expand Up @@ -136,47 +152,58 @@ echo "Placing binaries in: ${ARTIFACTS}"
mkdir -p "${ARTIFACTS}"
gh_endgroup

gh_group "Prepare buildx"
docker buildx use default
docker buildx create \
--platform linux/amd64,linux/arm64 \
--use \
--name nessiebuild \
--driver-opt network=host || docker buildx use nessiebuild
# Note: '--driver-opt network=host' is needed to be able to push to a local registry (e.g. localhost:5000)
gh_endgroup

gh_group "Docker buildx info"
docker buildx inspect
gh_endgroup
if [[ ${LOCAL} == 0 ]] ; then
gh_group "Prepare buildx"
${TOOL} buildx use default
${TOOL} buildx create \
--platform linux/amd64,linux/arm64 \
--use \
--name nessiebuild \
--driver-opt network=host || docker buildx use nessiebuild
# Note: '--driver-opt network=host' is needed to be able to push to a local registry (e.g. localhost:5000)
gh_endgroup

gh_group "Docker buildx info"
${TOOL} buildx inspect
gh_endgroup
fi

#
# Java multiplatform image
# Gradle Build
#

gh_group "Build Java linux/amd64"
gh_group "Build Java"
./gradlew "${GRADLE_PROJECT}:clean" "${GRADLE_PROJECT}:build" -x "${GRADLE_PROJECT}:check"
gh_endgroup

gh_group "Docker buildx build"
# All the platforms that are available
PLATFORMS="linux/amd64,linux/arm64/v8,linux/ppc64le,linux/s390x"
docker buildx build \
-f "${BASE_DIR}/tools/dockerbuild/docker/${DOCKERFILE}" \
--platform "${PLATFORMS}" \
-t "${IMAGE_NAME}:latest" \
-t "${IMAGE_NAME}:latest-java" \
-t "${IMAGE_NAME}:${IMAGE_TAG_BASE}" \
-t "${IMAGE_NAME}:${IMAGE_TAG_BASE}-java" \
"${BASE_DIR}/${PROJECT_DIR}" \
--push \
--provenance=false --sbom=false \
--output type=registry
# Note: '--output type=registry' is needed to be able to push to a local registry (e.g. localhost:5000)
# Note: '--provenance=false --sbom=false' work around UI issues in ghcr + quay showing 'unknown/unknown' architectures
gh_summary "## Java image tags, built for ${PLATFORMS}"
gh_summary "* \`docker pull ${IMAGE_NAME}:latest\`"
gh_summary "* \`docker pull ${IMAGE_NAME}:latest-java\`"
gh_summary "* \`docker pull ${IMAGE_NAME}:${IMAGE_TAG_BASE}\`"
gh_summary "* \`docker pull ${IMAGE_NAME}:${IMAGE_TAG_BASE}-java\`"
gh_endgroup
if [[ ${LOCAL} == 1 ]] ; then
gh_group "Docker build"
${TOOL} build \
--file "${BASE_DIR}/tools/dockerbuild/docker/${DOCKERFILE}" \
--tag "${IMAGE_NAME}:latest" \
--tag "${IMAGE_NAME}:${IMAGE_TAG_BASE}" \
"${BASE_DIR}/${PROJECT_DIR}"
gh_endgroup
else
gh_group "Docker buildx build"
# All the platforms that are available
PLATFORMS="linux/amd64,linux/arm64/v8,linux/ppc64le,linux/s390x"
${TOOL} buildx build \
--file "${BASE_DIR}/tools/dockerbuild/docker/${DOCKERFILE}" \
--platform "${PLATFORMS}" \
--tag "${IMAGE_NAME}:latest" \
--tag "${IMAGE_NAME}:latest-java" \
--tag "${IMAGE_NAME}:${IMAGE_TAG_BASE}" \
--tag "${IMAGE_NAME}:${IMAGE_TAG_BASE}-java" \
"${BASE_DIR}/${PROJECT_DIR}" \
--push \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a full review yet, but usually when I need to build a local image for use in minikube, I don't bother pushing to insecure registries.

Instead I remove this --push option and execute eval $(minikube docker-env) prior to building the image. This makes the build happen inside minikube, and the image is immediately available in the minikube node.

Have you tried this approach? Is it worth providing support for it as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too late :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difficulties begin when you use podman :(

Loads of issues with podman in minikube - that was the only approach that worked for me. (Yes, they document that you can use podman-remote and let that talk to minikube - but no - a huge API version mismatch... - just to name one annoyance)

--provenance=false --sbom=false \
--output type=registry
# Note: '--output type=registry' is needed to be able to push to a local registry (e.g. localhost:5000)
# Note: '--provenance=false --sbom=false' work around UI issues in ghcr + quay showing 'unknown/unknown' architectures
gh_summary "## Java image tags, built for ${PLATFORMS}"
gh_summary "* \`docker pull ${IMAGE_NAME}:latest\`"
gh_summary "* \`docker pull ${IMAGE_NAME}:latest-java\`"
gh_summary "* \`docker pull ${IMAGE_NAME}:${IMAGE_TAG_BASE}\`"
gh_summary "* \`docker pull ${IMAGE_NAME}:${IMAGE_TAG_BASE}-java\`"
gh_endgroup
fi