Skip to content

Commit

Permalink
feat: split installer and imager images
Browse files Browse the repository at this point in the history
The `installer` image is used for initial install and upgrades, so it
contains only Talos artifacts for the same arch as the image

The `imager` image is used to produce Talos images, so it contains artifacts
for all arhitectures, so that it's possible e.g. to build Raspberry PI
Talos image on amd64 machine

Unpacked size comparison for amd64 (before this change, size of
`installer` was same as `imager` size):

```
REPOSITORY                               TAG                                      IMAGE ID       CREATED         SIZE
ghcr.io/smira/installer                  v0.14.0-alpha.2-8-g73293bc2-dirty        264995cf56df   4 minutes ago   176MB
ghcr.io/smira/imager                     v0.14.0-alpha.2-8-g73293bc2-dirty        3ba8e0176565   6 minutes ago   616MB
```

As the `installer` image is pulled into `tmpfs` on upgrade, it's important
to have it as small as possible to avoid putting too much pressure on
node memory.

Both images are generated anyways for both `amd64` and `arm64` target
architectures.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
  • Loading branch information
smira committed Dec 3, 2021
1 parent 1a13aaa commit 2a0da06
Show file tree
Hide file tree
Showing 4 changed files with 40 additions and 29 deletions.
2 changes: 1 addition & 1 deletion .drone.jsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ local Pipeline(name, steps=[], depends_on=[], with_docker=true, disable_clone=fa

local generate = Step("generate", target="generate docs", depends_on=[setup_ci]);
local check_dirty = Step("check-dirty", depends_on=[generate]);
local build = Step("build", target="talosctl-linux talosctl-darwin talosctl-windows kernel initramfs installer talos", depends_on=[check_dirty], environment={"IMAGE_REGISTRY": local_registry, "PUSH": true});
local build = Step("build", target="talosctl-linux talosctl-darwin talosctl-windows kernel initramfs installer imager talos", depends_on=[check_dirty], environment={"IMAGE_REGISTRY": local_registry, "PUSH": true});
local lint = Step("lint", depends_on=[build]);
local talosctl_cni_bundle = Step('talosctl-cni-bundle', depends_on=[build, lint]);
local iso = Step("iso", target="iso", depends_on=[build], environment={"IMAGE_REGISTRY": local_registry});
Expand Down
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,8 @@ ONBUILD RUN find /rootfs \
&& rm -rf /initramfs
ONBUILD WORKDIR /

FROM installer AS imager

# The test target performs tests on the source code.

FROM base AS unit-tests-runner
Expand Down
51 changes: 23 additions & 28 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ COMMON_ARGS += --build-arg=https_proxy=$(https_proxy)

CI_ARGS ?=

all: initramfs kernel installer talosctl talosctl-image talos
all: initramfs kernel installer imager talosctl talosctl-image talos

# Help Menu

Expand Down Expand Up @@ -196,15 +196,20 @@ initramfs: ## Builds the compressed initramfs and outputs it to the artifact dir
@$(MAKE) local-$@ DEST=$(ARTIFACTS) PUSH=false TARGET_ARGS="--allow security.insecure"

.PHONY: installer
installer: ## Builds the container image for the installer and outputs it to the artifact directory.
installer: ## Builds the container image for the installer and outputs it to the registry.
@INSTALLER_ARCH=targetarch \
$(MAKE) registry-$@ TARGET_ARGS="--allow security.insecure"

.PHONY: imager
imager: ## Builds the container image for the imager and outputs it to the registry.
@$(MAKE) registry-$@ TARGET_ARGS="--allow security.insecure"

.PHONY: talos
talos: ## Builds the Talos container image and outputs it to the artifact directory.
talos: ## Builds the Talos container image and outputs it to the registry.
@$(MAKE) registry-$@ TARGET_ARGS="--allow security.insecure"

.PHONY: talosctl-image
talosctl-image: ## Builds the talosctl container image and outputs it to the artifact directory.
talosctl-image: ## Builds the talosctl container image and outputs it to the registry.
@$(MAKE) registry-talosctl TARGET_ARGS="--allow security.insecure"

talosctl-%:
Expand All @@ -213,26 +218,26 @@ talosctl-%:
talosctl: $(TALOSCTL_DEFAULT_TARGET) ## Builds the talosctl binary for the local machine.

image-%: ## Builds the specified image. Valid options are aws, azure, digital-ocean, gcp, and vmware (e.g. image-aws)
@docker pull $(REGISTRY_AND_USERNAME)/installer:$(IMAGE_TAG)
@docker pull $(REGISTRY_AND_USERNAME)/imager:$(IMAGE_TAG)
@for platform in $(subst $(,),$(space),$(PLATFORM)); do \
arch=`basename "$${platform}"` ; \
docker run --rm -v /dev:/dev --privileged $(REGISTRY_AND_USERNAME)/installer:$(IMAGE_TAG) image --platform $* --arch $$arch --tar-to-stdout | tar xz -C $(ARTIFACTS) ; \
docker run --rm -v /dev:/dev --privileged $(REGISTRY_AND_USERNAME)/imager:$(IMAGE_TAG) image --platform $* --arch $$arch --tar-to-stdout | tar xz -C $(ARTIFACTS) ; \
done

images: image-aws image-azure image-digital-ocean image-gcp image-hcloud image-metal image-nocloud image-openstack image-scaleway image-upcloud image-vmware image-vultr ## Builds all known images (AWS, Azure, DigitalOcean, GCP, HCloud, Metal, NoCloud, Openstack, Scaleway, UpCloud, Vultr and VMware).

sbc-%: ## Builds the specified SBC image. Valid options are rpi_4, rock64, bananapi_m64, libretech_all_h3_cc_h5, rockpi_4 and pine64 (e.g. sbc-rpi_4)
@docker pull $(REGISTRY_AND_USERNAME)/installer:$(IMAGE_TAG)
@docker run --rm -v /dev:/dev --privileged $(REGISTRY_AND_USERNAME)/installer:$(IMAGE_TAG) image --platform metal --arch arm64 --board $* --tar-to-stdout | tar xz -C $(ARTIFACTS)
@docker pull $(REGISTRY_AND_USERNAME)/imager:$(IMAGE_TAG)
@docker run --rm -v /dev:/dev --privileged $(REGISTRY_AND_USERNAME)/imager:$(IMAGE_TAG) image --platform metal --arch arm64 --board $* --tar-to-stdout | tar xz -C $(ARTIFACTS)

sbcs: sbc-rpi_4 sbc-rock64 sbc-bananapi_m64 sbc-libretech_all_h3_cc_h5 sbc-rockpi_4 sbc-pine64 ## Builds all known SBC images (Raspberry Pi 4 Model B, Rock64, Banana Pi M64, Radxa ROCK Pi 4, pine64, and Libre Computer Board ALL-H3-CC).

.PHONY: iso
iso: ## Builds the ISO and outputs it to the artifact directory.
@docker pull $(REGISTRY_AND_USERNAME)/installer:$(IMAGE_TAG)
@docker pull $(REGISTRY_AND_USERNAME)/imager:$(IMAGE_TAG)
@for platform in $(subst $(,),$(space),$(PLATFORM)); do \
arch=`basename "$${platform}"` ; \
docker run --rm -e SOURCE_DATE_EPOCH=$(SOURCE_DATE_EPOCH) -i $(REGISTRY_AND_USERNAME)/installer:$(IMAGE_TAG) iso --arch $$arch --tar-to-stdout | tar xz -C $(ARTIFACTS) ; \
docker run --rm -e SOURCE_DATE_EPOCH=$(SOURCE_DATE_EPOCH) -i $(REGISTRY_AND_USERNAME)/imager:$(IMAGE_TAG) iso --arch $$arch --tar-to-stdout | tar xz -C $(ARTIFACTS) ; \
done

.PHONY: boot
Expand Down Expand Up @@ -352,26 +357,15 @@ $(ARTIFACTS)/$(TALOS_RELEASE): $(ARTIFACTS)/$(TALOS_RELEASE)/vmlinuz $(ARTIFACTS
# download release artifacts for specific version
$(ARTIFACTS)/$(TALOS_RELEASE)/%:
@mkdir -p $(ARTIFACTS)/$(TALOS_RELEASE)/
@case "$(TALOS_RELEASE)" in \
v0.6*) \
curl -L -o "$(ARTIFACTS)/$(TALOS_RELEASE)/$*" "https://github.com/talos-systems/talos/releases/download/$(TALOS_RELEASE)/$*" \
;; \
v0.7.0-alpha.[0-3]) \
curl -L -o "$(ARTIFACTS)/$(TALOS_RELEASE)/$*" "https://github.com/talos-systems/talos/releases/download/$(TALOS_RELEASE)/$*" \
@case "$*" in \
vmlinuz) \
curl -L -o "$(ARTIFACTS)/$(TALOS_RELEASE)/$*" "https://github.com/talos-systems/talos/releases/download/$(TALOS_RELEASE)/vmlinuz-amd64" \
;; \
*) \
case "$*" in \
vmlinuz) \
curl -L -o "$(ARTIFACTS)/$(TALOS_RELEASE)/$*" "https://github.com/talos-systems/talos/releases/download/$(TALOS_RELEASE)/vmlinuz-amd64" \
;; \
initramfs.xz) \
curl -L -o "$(ARTIFACTS)/$(TALOS_RELEASE)/$*" "https://github.com/talos-systems/talos/releases/download/$(TALOS_RELEASE)/initramfs-amd64.xz" \
;; \
esac \
initramfs.xz) \
curl -L -o "$(ARTIFACTS)/$(TALOS_RELEASE)/$*" "https://github.com/talos-systems/talos/releases/download/$(TALOS_RELEASE)/initramfs-amd64.xz" \
;; \
esac


.PHONY: release-artifacts
release-artifacts:
@for release in $(RELEASES); do \
Expand All @@ -394,12 +388,13 @@ ifeq ($(DOCKER_LOGIN_ENABLED), true)
@docker login --username "$(GHCR_USERNAME)" --password "$(GHCR_PASSWORD)" $(IMAGE_REGISTRY)
endif

push: login ## Pushes the installer, and talos images to the configured container registry with the generated tag.
push: login ## Pushes the installer, imager, talos and talosctl images to the configured container registry with the generated tag.
@$(MAKE) installer PUSH=true
@$(MAKE) imager PUSH=true
@$(MAKE) talos PUSH=true
@$(MAKE) talosctl-image PUSH=true

push-%: login ## Pushes the installer, and talos images to the configured container registry with the specified tag (e.g. push-latest).
push-%: login ## Pushes the installer, imager, talos and talosctl images to the configured container registry with the specified tag (e.g. push-latest).
@$(MAKE) push IMAGE_TAG=$*

.PHONY: clean
Expand Down
14 changes: 14 additions & 0 deletions hack/release.toml
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,20 @@ A set of Talos ehancements is going to unlock a number of exciting features in t
title = "NTP Sync"
description = """\
Talos NTP sync process was improved to align better with kernel time adjustment periods and to filter out spikes.
"""

[notes.installer]
title = "`installer` and `imager` images"
description = """\
Talos supports two target architectures: `amd64` and `arm64`, so all Talos images are built for both `amd64` and `arm64`.
New image `imager` was added which contains Talos assets for both architectures which allows to generate Talos disk images
cross-arch: e.g. generate Talos Raspberry PI disk image on `amd64` machine.
As `installer` image is used only to do initial install and upgrades, it now contains Talos assets for a specific architecture.
This reduces size of the `installer` image leading to faster upgrades and less memory usage.
There are no user-visible changes except that now `imager` container image should be used to produce Talos disk images.
"""

[make_deps]
Expand Down

0 comments on commit 2a0da06

Please sign in to comment.