diff --git a/Makefile b/Makefile index 7a81ba94eb..cc82f9d56e 100644 --- a/Makefile +++ b/Makefile @@ -172,7 +172,7 @@ build-examples: ## Build all of the example packages @test -s ./build/zarf-package-component-webhooks-$(ARCH)-0.0.1.tar.zst || $(ZARF_BIN) package create examples/component-webhooks -o build -a $(ARCH) --confirm build-injector-linux: ## Build the Zarf injector for AMD64 and ARM64 - docker run --rm --user "$(id -u)":"$(id -g)" -v $$PWD/src/injector:/usr/src/zarf-injector -w /usr/src/zarf-injector rust:1.71.0-bookworm make build-injector-linux + docker run --rm --user "$(id -u)":"$(id -g)" -v $$PWD/src/injector:/usr/src/zarf-injector -w /usr/src/zarf-injector rust:1.71.0-bookworm make build-injector-linux list-sizes ## NOTE: Requires an existing cluster or the env var APPLIANCE_MODE=true .PHONY: test-e2e diff --git a/site/src/content/docs/faq.mdx b/site/src/content/docs/faq.mdx index 1e662a262a..e4bca91bba 100644 --- a/site/src/content/docs/faq.mdx +++ b/site/src/content/docs/faq.mdx @@ -24,34 +24,6 @@ No, the Zarf binary and init package can be downloaded from the [Releases Page]( Zarf is statically compiled and written in [Go](https://golang.org/) and [Rust](https://www.rust-lang.org/), so it has no external dependencies. For Linux, Zarf can bring a Kubernetes cluster using [K3s](https://k3s.io/). For Mac and Windows, Zarf can leverage any available local or remote cluster the user has access to. Currently, the K3s installation Zarf performs does require a [Systemd](https://en.wikipedia.org/wiki/Systemd) based system and `root` (not just `sudo`) access. -## What is the Zarf Agent? - -The Zarf Agent is a [Kubernetes Mutating Webhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook) that is installed into the cluster during `zarf init`. The Agent is responsible for modifying [Kubernetes PodSpec](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec) objects [Image](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Container.Image) fields to point to the Zarf Registry. This allows the cluster to pull images from the Zarf Registry instead of the internet without having to modify the original image references. The Agent also modifies [Flux GitRepository](https://fluxcd.io/docs/components/source/gitrepositories/) objects to point to the local Git Server. - -## Why doesn't the Zarf Agent create secrets it needs in the cluster? - -During early discussions and [subsequent decision](https://github.com/defenseunicorns/zarf/blob/main/adr/0005-mutating-webhook.md) to use a Mutating Webhook, we decided to not have the Agent create any secrets in the cluster. This is to avoid the Agent having to have more privileges than it needs as well as to avoid collisions with Helm. The Agent today simply responds to requests to patch PodSpec and GitRepository objects. - -The Agent does not need to create any secrets in the cluster. Instead, during `zarf init` and `zarf package deploy`, secrets are automatically created as [Helm Postrender Hook](https://helm.sh/docs/topics/advanced/#post-rendering) for any namespaces Zarf sees. If you have resources managed by [Flux](https://fluxcd.io/) that are not in a namespace managed by Zarf, you can either create the secrets manually or include a manifest to create the namespace in your package and let Zarf create the secrets for you. - -## How can a Kubernetes resource be excluded from the Zarf Agent? - -Resources can be excluded at the namespace or resources level by adding the `zarf.dev/agent: ignore` label. - -## What happens to resources that exist in the cluster before `zarf init`? - -During the [`zarf init`](/commands/zarf_init) operation, the Zarf Agent will patch any existing namespaces with the `zarf.dev/agent: ignore` label to prevent the Agent from modifying any resources in that namespace. This is done because there is no way to guarantee the images used by pods in existing namespaces are available in the Zarf Registry. - -If you would like to adopt pre-existing resources into a Zarf deployment you can use the `--adopt-existing-resources` flag on [`zarf package deploy`](/commands/zarf_package_deploy/) to adopt those resources into the Helm Releases that Zarf manages (including namespaces). This will add the requisite annotations and labels to those resources and drop the `zarf.dev/agent: ignore` label from any namespaces specified by those resources. - -:::note - -Zarf will refuse to adopt the Kubernetes [initial namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#initial-namespaces). It is recommended that you do not deploy resources into the `default` or `kube-*` namespaces with Zarf. - -Additionally, when adopting resources, you should ensure that the namespaces you are adopting are dedicated to Zarf, or that you go back and manually add the `zarf.dev/agent: ignore` label to any non-Zarf managed resources in those namespaces (and ensure that updates to those resources do not strip that label) otherwise you may see [ImagePullBackOff](https://kubernetes.io/docs/concepts/containers/images/#imagepullbackoff) errors. - -::: - ## How can I improve the speed of loading large images from Docker on `zarf package create`? Due to some limitations with how Docker provides access to local image layers, `zarf package create` has to rely on `docker save` under the hood which is [very slow overall](https://github.com/defenseunicorns/zarf/issues/1214) and also takes a long time to report progress. We experimented with many ways to improve this, but for now recommend leveraging a local docker registry to speed up the process. diff --git a/site/src/content/docs/ref/init-package.mdx b/site/src/content/docs/ref/init-package.mdx index 660e04e290..c2f98e1e6e 100644 --- a/site/src/content/docs/ref/init-package.mdx +++ b/site/src/content/docs/ref/init-package.mdx @@ -5,110 +5,121 @@ sidebar: --- import Mermaid from "@components/Mermaid.astro"; +import Details from "@components/Details.astro"; -The 'init' package is a special Zarf Package (denoted by `kind: ZarfInitConfig` in its `zarf.yaml`) that initializes a cluster with the requisite air gap services when running `zarf init`. This allows future Zarf Packages to store any required resources (i.e. container images and git repositories) so that they can be retrieved later. +In a traditional Kubernetes deployment, clusters pull resources (e.g. cluster images, OCI artifacts, Git repos) from external sources. -The default 'init' package that Zarf ships is defined in the `zarf.yaml` that lives at the [root of the Zarf repository](https://github.com/defenseunicorns/zarf/blob/main/zarf.yaml), and is constructed from composed components that provide a foundation for customization. If you would like to change the behavior of the 'init' package you can do so by modifying this `zarf.yaml` or any of the composed components that it references and running `zarf package create` at the root of the repository. You can learn more about creating a custom init package in the [Creating a Custom 'init' Package Tutorial](/tutorials/8-custom-init-packages). +However, in an air-gapped environment, these external providers are not available, or exist at different locations to their references within Kubernetes manifests. -Upon deployment, the init package creates a `zarf` namespace within your K8s cluster and deploys pods, services, and secrets to that namespace based on the components selected for deployment. +Zarf solves this problem with its 'init' package, a special Zarf Package (traditionally deployed first) that provides the necessary mechanisms to enable air-gapped Kubernetes, and deliver DevSecOps across air-gaps. -## Required Component +:::note[tldr;] -Zarf's mutation capabilities require that the [`zarf-agent`](/faq#what-is-the-zarf-agent) component of the init package is deployed and active within the cluster, meaning that it cannot be disabled and is always running. This component intercepts requests to create resources and uses the `zarf-state` secret to mutate them to point to their air gap equivalents. It is automatically deployed whenever a `zarf init` command is executed. +Don't care about the details and just want to get deploying as quickly as possible? Run the following after connecting to a cluster: -| Component | Description | -| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -| zarf-agent | A Kubernetes mutating webhook installed during `zarf init` that converts Pod specs and Flux GitRepository objects to match their air gap equivalents. | +```bash +zarf tools download-init +zarf init --confirm +``` -:::note +Want to see a guided `zarf init`? Check out the [Zarf Init tutorial](/tutorials/1-initializing-a-k8s-cluster/). -The `zarf-agent` will mutate any resources it sees that have not already been patched and don't have the `zarf.dev/agent: ignore` label applied. This label is automatically applied to all namespaces that exist prior to `zarf init`, and will prevent the `zarf-agent` from mutating system resources. You can manually apply this label to any additional namespaces or resources after the fact to prevent the `zarf-agent` from acting upon them. See the FAQ entry to learn more about [what happens to resources that were deployed prior to `zarf init`](/faq#what-happens-to-resources-that-exist-in-the-cluster-before-zarf-init). +View all init options w/ [`zarf init --help`](/commands/zarf_init/). ::: ## Core Components -In addition to the required `zarf-agent` component, Zarf also offers components that provide additional functionality and can be enabled as needed based on your desired end-state. +An 'init' package requires a series of specially named, and configured components to ensure the cluster is correctly initialized. These components are: -In most scenarios, Zarf will also deploy an internal registry using the three components described below. However, Zarf can be configured to use an already existing registry with the `--registry-*` flags when running `zarf init` (detailed information on all `zarf init` command flags can be found in the [zarf init CLI](/commands/zarf_init/) section). This option skips the injector and seed process, and will not deploy a registry inside of the cluster. Instead, it uploads any images to the externally configured registry. +- [`zarf-injector`](#zarf-injector) +- [`zarf-seed-registry`](#zarf-seed-registry) +- [`zarf-registry`](#zarf-registry) +- [`zarf-agent`](#zarf-agent) -| Components | Description | -| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -| zarf-injector | Adds a Rust binary to the working directory to be injected into the cluster during registry bootstrapping. | -| zarf-seed-registry | Adds a temporary container registry so Zarf can bootstrap itself into the cluster. | -| zarf-registry | Adds a long-lived container registry service—[docker registry](https://docs.docker.com/registry/)—into the cluster. | +### `zarf-injector` and `zarf-seed-registry` -:::note +One of the most challenging aspects of deploying into an air-gapped environment is the initial bootstrapping of the cluster. -Given the registry is a core part of any Kubernetes deployment you MUST either specify an external registry with the `--registry-*` flags or use the injected registry which is why it cannot be selected with `--components` like the components below. +A cluster needs a registry to pull images from, but spinning up a registry requires an image to be pulled from a registry - chicken, meet egg. -::: +To ensure that our approach is distro-agnostic, the Zarf team developed a unique solution to seed the container registry into the cluster, populate said registry, and redirect cluster resources to use the air-gapped registry. -:::note +Shoving random data into a cluster is generally a bad idea, and an antipattern overall to containerization. However in the case of Zarf, and air-gapped environments, certain liberties must be taken. -The Zarf Registry is initially injected as a series of config maps that bootstraps itself into the cluster and then binds to a NodePort to allow the kubelet to pull images and setup the final registry deployment. Doing this keeps Zarf cluster agnostic however does require that the kubelet be able to reach out to a cluster NodePort service which may require changes to firewall configurations like allowing UDP traffic between nodes if using something like VXLAN tunneling. +While there is no distro-agnostic method to inject images into a cluster, every cluster has support for `configmaps`. However, the size of a `configmap` is limited to 1MB (technically only limited by whatever is configured in `etcd`, the default is 1MB), and the `registry:2` image is around 10MB (as of this writing). So we split the `registry:2` image into chunks and inject them into the cluster as `configmaps`. -::: +But then we have another problem of how to reassemble the image on the other side, as we don't have any consistent image that exists in the cluster that would have such utilities. This is where the `zarf-injector` Rust binary comes in. -:::tip +> For compiling the `zarf-injector` binary, refer to its [README.md](https://github.com/defenseunicorns/zarf/tree/main/src/injector/README.md). -You can further customize how the registry behaves by setting variables such as `REGISTRY_PVC_SIZE` with a [config file](/ref/config-files/) or `--set` on `zarf init`. +The `zarf-injector` binary is statically compiled and injected into the cluster as a `configmap` along with the chunks of the `registry:2` image. During the `zarf-seed-registry`'s deployment, the `zarf-injector` binary is run in a pod that mounts the `configmaps` and reassembles the `registry:2` image. It then hosts a temporary, pull-only Docker registry implemented in Rust so that a real registry can be deployed into the cluster from the hosted `registry:2` image. -To see a full list of `variables` you can view the [zarf.yaml that defines the registry](https://github.com/defenseunicorns/zarf/blob/main/packages/zarf-registry/zarf.yaml). +> While the `zarf-injector` component *must* be defined and deployed *before* the `zarf-seed-registry` component, the magic doesn't start until `zarf-seed-registry` is deployed. The `zarf-injector` component for the most part just copies the `zarf-injector` binary to `###ZARF_TEMP###/zarf-injector`. -::: +When `zarf init` deploys the `zarf-seed-registry` component, the following happens: -Beyond the registry, their are also fully-optional components available for the init package. Many of these also have external configurations you can set with `zarf init` (such as `--git-*`), but these components provide an easy way to get started in environments where these core services are needed and may not already exist. +1. Zarf injects the `zarf-injector` binary and the `registry:2` image chunks into the cluster. +2. Zarf connects to the cluster and grabs a pod that is running an image that is already present in the cluster. +3. Zarf spins up a pod using the existing image, mounts the `configmaps` that contain the `zarf-injector` binary and the `registry:2` image chunks and runs the `zarf-injector` binary. -| Components | Description | -| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| k3s | REQUIRES ROOT (not sudo). Installs a lightweight Kubernetes Cluster on the local host—[K3s](https://k3s.io/)—and configures it to start up on boot. | -| logging | Adds a log monitoring stack—[promtail/loki/grafana (aka PLG)](https://github.com/grafana/loki)—into the cluster. | -| git-server | Adds a [GitOps](https://about.gitlab.com/topics/gitops/)-compatible source control service—[Gitea](https://gitea.io/en-us/)—into the cluster. | +:::note -There are two ways to deploy these optional components. First, you can provide a comma-separated list of components to the `--components` flag, such as `zarf init --components k3s,git-server --confirm`, or, you can choose to exclude the `--components` and `--confirm` flags and respond with a yes (`y`) or no (`n`) for each optional component when interactively prompted. +Doing this keeps Zarf cluster agnostic, however does require that the kubelet be able to reach out to a cluster NodePort service, which may require changes to firewall configurations like allowing UDP traffic between nodes if using something like VXLAN tunneling. -:::caution +::: -(Linux only) Deploying the 'k3s' component will require `root` access (not just `sudo`), as it modifies your host machine to install the cluster and by default only provides access to the cluster to the `root` user. +4. The `zarf-injector` binary reassembles the `registry:2` image and hosts a temporary registry that the cluster can pull from. +5. The `docker-registry` chart in the `zarf-seed-registry` component is then deployed, with its `image.repository` set to the temporary registry that the `zarf-injector` binary is hosting (consumed as the `###ZARF_SEED_REGISTRY###` built-in variable set at runtime). +6. Once the `docker-registry` chart is deployed, the `zarf-seed-registry` component is marked as complete and the `zarf-injector` pod is removed from the cluster. +7. Deployment proceeds to the `zarf-registry` component. + +:::note + +The `registry:2` image and the Zarf Agent image can be configured with a custom init package using the `registry_image_*` and `agent_image_*` templates defined in the Zarf repo's [zarf-config.toml](https://github.com/defenseunicorns/zarf/blob/main/zarf-config.toml). This allows you to swap them for enterprise provided / hardened versions if desired such as those provided by [Iron Bank](https://repo1.dso.mil/dsop/opensource/defenseunicorns/zarf/zarf-agent). ::: -:::tip +### `zarf-registry` -The `k3s` component included in Zarf differs from the default `k3s` install in that it disables the installation of `traefik` out of the box. This was done so that people could more intentionally choose if they wanted `traefik` or another ingress provider (or no ingress at all) depending on their needs. If you would like to return `k3s` to its defaults, you can set the `K3S_ARGS` zarf variable to an empty string: +The `zarf-registry` component is a long-lived container registry service that is deployed into the cluster. -```text -root@machine ~ # zarf init --components k3s --set K3S_ARGS="" --confirm -``` +It leverages the same `docker-registry` chart used in `zarf-seed-registry` but with a few key differences: -::: +1. The `image.repository` is set to the value of the built-in variable `###ZARF_REGISTRY###` which is set at runtime to the registry hosted by `zarf-seed-registry`. +{/* you know, why DO we do this? if we kept the repository the same, and kept running the injector, couldnt this handle cluster full-deaths??? (ofc would need to tweak the zarf-agent to not mutate the image.repository for the docker-registry chart under the zarf namespace, but i think it might be doable) */} +2. A `connect` manifest for running [`zarf connect registry`](/commands/zarf_connect_registry/) to tunnel to the Zarf Registry. +3. A configmap to satisfy [KEP-1755](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry) :::tip -You can further customize how the git-server behaves by setting variables such as `GIT_SERVER_PVC_SIZE` with a [config file](/ref/config-files/) or `--set` on `zarf init`. +You can further customize how the registry behaves by setting variables such as `REGISTRY_PVC_SIZE` with a [config file](/ref/config-files/) or `--set` on `zarf init`. -To see a full list of `variables` you can view the [zarf.yaml that defines the git-server](https://github.com/defenseunicorns/zarf/blob/main/packages/gitea/zarf.yaml). +To see a full list of `variables` you can view the [`zarf.yaml` that defines the registry](https://github.com/defenseunicorns/zarf/blob/main/packages/zarf-registry/zarf.yaml). ::: -## What Makes the Init Package Special +#### Using External Registries -Deploying into air gapped environments is a hard problem, particularly when the K8s environment doesn't have a container registry for you to store images in already. This results in a dilemma where the container registry image must be introduced to the cluster, but there is no container registry to push it to as the image is not yet in the cluster - chicken, meet egg. To ensure that our approach is distro-agnostic, we developed a unique solution to seed the container registry into the cluster. +Zarf can be configured to use an already existing registry with the `--registry-*` flags when running [`zarf init`](/commands/zarf_init/). -This is done with the `zarf-injector` [component](https://github.com/defenseunicorns/zarf/blob/main/packages/zarf-injector/zarf.yaml) which injects a single rust binary (statically compiled) and a series of configmap chunks of a `registry:2` image into an ephemeral pod that is based on an existing image in the cluster. This gives us a running registry to bootstrap from and deploy the rest of the 'init' package and any other packages down the line. +This option skips the injector and seed process, and will not deploy a registry inside of the cluster. Instead, it pushes any images contained in the package to the externally configured registry. :::note -The `registry:2` image and the Zarf Agent image can be configured with a custom init package using the `registry_image_*` and `agent_image_*` templates defined in the Zarf repo's [zarf-config.toml](https://github.com/defenseunicorns/zarf/blob/main/zarf-config.toml). This allows you to swap them for enterprise provided / hardened versions if desired such as those provided by [Iron Bank](https://repo1.dso.mil/dsop/opensource/defenseunicorns/zarf/zarf-agent). +Given the registry is a core part of any Kubernetes deployment you MUST either specify an external registry with the `--registry-*` flags or use the injected registry. ::: -## Making the Registry Highly-Available +#### Making the Registry Highly-Available -By default, the registry included in the init package creates a ReadWriteOnce PVC and is only scheduled to run on one node at a time. This setup is usually enough for smaller and simpler deployments. However, for larger deployments or those where nodes are frequently restarted or updated, you may want to make the registry highly-available. +By default, the registry included in the init package creates a `ReadWriteOnce` PVC and is only scheduled to run on one node at a time. -This approach requires certain prerequisites, such as a storage class that supports ReadWriteMany, or being in an environment that allows you to configure the registry to use an S3-compatible backend. Additionally, you must provide custom configuration to the registry to ensure it is distributed across all nodes and has the appropriate number of replicas. Below is an example [configuration file](/ref/config-files/) using a ReadWriteMany storage class: +This setup is usually enough for smaller and simpler deployments. However, for larger deployments or those where nodes are frequently restarted or updated, you may want to make the registry highly-available. + +This approach requires certain prerequisites, such as a storage class that supports `ReadWriteMany`, or being in an environment that allows you to configure the registry to use an S3-compatible backend. + +Additionally, you must provide custom configuration to the registry to ensure it is distributed across all nodes and has the appropriate number of replicas. Below is an example [configuration file](/ref/config-files/) using a ReadWriteMany storage class: ```yaml # zarf-config.yaml @@ -134,58 +145,164 @@ package: Notably, the `REGISTRY_AFFINITY_CUSTOM` variable overrides the default pod anti-affinity, and `REGISTRY_HPA_AUTO_SIZE` automatically adjusts the minimum and maximum replicas for the registry based on the number of nodes in the cluster. If you prefer to manually set the minimum and maximum replicas, you can use `REGISTRY_HPA_MIN` and `REGISTRY_HPA_MAX` to specify the desired values. -## The `zarf init` Lifecycle - -The `zarf init` lifecycle is _very similar_ to the [`zarf package deploy` lifecycle](/ref/deploy/) except that it sets up resources specific to Zarf such as the `zarf-state` and performs special actions such as the injection procedure. - -B2 - B2(handle multipart package)-->B3 - B3(extract archive to temp dir)-->B4 - B4(validate package checksums and signature)-->B5 - B5(filter components by architecture & OS)-->B6 - B6(save SBOM files to current dir)-->B7 - B7(handle deprecations and breaking changes)-->B9 - B9(confirm package deploy):::prompt-->B10 - B10(process deploy-time variables)-->B11 - B11(prompt for missing variables)-->B12 - B12(prompt to confirm components)-->B13 - B13(prompt to choose components in '.group')-->B14 - - subgraph "" - B52 --> |Yes|B14(deploy each component)-->B14 - B14 --> B15{Component is \n zarf-seed-registry} - B15 --> |Yes|B51(initialize zarf-state secret):::action - B51 --> B52{External \n registry configured} - B52 --> |No|B53(run injection process):::action-->B16 - B15 --> |No|B16(run each '.actions.onDeploy.before'):::action-->B16 - B16 --> B17(copy '.files')-->B18 - B18(load Zarf State)-->B19 - B19(push '.images')-->B20 - B20(push '.repos')-->B21 - B21(process '.dataInjections')-->B22 - B22(install '.charts')-->B23 - B23(apply '.manifests')-->B24 - B24(run each '.actions.onDeploy.after'):::action-->B24 - B24-->B25{Success?} - B25-->|Yes|B26(run each\n'.actions.onDeploy.success'):::action-->B26 - B25-->|No|B27(run each\n'.actions.onDeploy.failure'):::action-->B27-->B999 - - B999[Abort]:::fail - end - - B26-->B28(print Zarf connect table) - B28-->B29(save package data to cluster) - - - classDef prompt fill:#4adede,color:#000000 - classDef action fill:#bd93f9,color:#000000 - classDef fail fill:#aa0000 -`}/> +### `zarf-agent` + +{/* TODO: document and flesh out how the mutations operate for the agent */} + +The `zarf-agent` is a [Kubernetes Mutating Webhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook) that intercepts requests to create resources and uses the `zarf-state` secret to mutate them to point to their air-gapped equivalents. + +The `zarf-agent` is responsible for modifying [Kubernetes PodSpec](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec) objects [Image](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Container.Image) fields to point to the Zarf Registry. + +This allows the cluster to pull images from the Zarf Registry instead of the internet without having to modify the original image references. + +The `zarf-agent` also modifies [Flux GitRepository](https://fluxcd.io/docs/components/source/gitrepositories/) objects to point to the local Git Server. + +> Support for mutating `Application` and `Repository` objects in ArgoCD is in [`beta`](/roadmap#beta) and should tested on non-production clusters before being deployed to production clusters. + +:::note + +During the [`zarf init`](/commands/zarf_init) operation, the Zarf Agent will add the `zarf.dev/agent: ignore` label to prevent the Agent from modifying any resources in that namespace. This is done because there is no way to guarantee the images used by pods in existing namespaces are available in the Zarf Registry. + +If you would like to adopt pre-existing resources into a Zarf deployment you can use the `--adopt-existing-resources` flag on [`zarf package deploy`](/commands/zarf_package_deploy/) to adopt those resources into the Helm Releases that Zarf manages (including namespaces). This will add the requisite annotations and labels to those resources and drop the `zarf.dev/agent: ignore` label from any namespaces specified by those resources. + +::: + +#### Excluding Resources from `zarf-agent` + +Resources can be excluded at the namespace or resources level by adding the `zarf.dev/agent: ignore` label. + +Zarf will refuse to adopt the Kubernetes [initial namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#initial-namespaces) (`default`, `kube-*`, etc...). This is because these namespaces are critical to the operation of the cluster and should not be managed by Zarf. + +Additionally, when adopting resources, ensure that the namespaces specified are dedicated to Zarf, or add the `zarf.dev/agent: ignore` label to any non-Zarf managed resources in those namespaces (and ensure that updates to those resources do not strip that label) otherwise [ImagePullBackOff](https://kubernetes.io/docs/concepts/containers/images/#imagepullbackoff) errors may occur. + +The Agent does not need to create any secrets in the cluster. Instead, during `zarf init` and `zarf package deploy`, secrets are automatically created in a [Helm Postrender Hook](https://helm.sh/docs/topics/advanced/#post-rendering) for any namespaces Zarf sees. If you have resources managed by [Flux](https://fluxcd.io/) that are not in a namespace managed by Zarf, you can either create the secrets manually or include a manifest to create the namespace in your package and let Zarf create the secrets for you. + +## Optional Components + +The Zarf team maintains some optional components in the default 'init' package. + +| Components | Description | +| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| k3s | REQUIRES ROOT (not sudo). Installs a lightweight Kubernetes Cluster on the local host [K3s](https://k3s.io/) and configures it to start up on boot. | +| logging | Adds a log monitoring stack [promtail/loki/grafana (aka PLG)](https://github.com/grafana/loki) into the cluster. | +| git-server | Adds a [GitOps](https://about.gitlab.com/topics/gitops/)-compatible source control service [Gitea](https://gitea.io/en-us/) into the cluster. | + +There are two ways to deploy these optional components. First, you can provide a comma-separated list of components to the `--components` flag, such as `zarf init --components k3s,git-server --confirm`, or, you can choose to exclude the `--components` and `--confirm` flags and respond with a yes (`y`) or no (`n`) for each optional component when interactively prompted. + +:::caution + +(Linux only) Deploying the 'k3s' component will require `root` access (not just `sudo`), as it modifies your host machine to install the cluster and by default only provides access to the cluster to the `root` user. + +::: :::tip -Given that these flows are so similar you actually can `zarf package deploy` the init package. This is similar to accepting the defaults for any init-specific command flags and is useful when trying to deploy packages in a more generic way. +The `k3s` component included differs from the default `k3s` install in that it disables the installation of `traefik` out of the box. This was done so that people could more intentionally choose if they wanted `traefik` or another ingress provider (or no ingress at all) depending on their needs. If you would like to return `k3s` to its defaults, you can set the `K3S_ARGS` Zarf variable to an empty string: + +```text +root@machine ~ # zarf init --components k3s --set K3S_ARGS="" --confirm +``` + +::: + +:::tip + +You can further customize how the git-server behaves by setting variables such as `GIT_SERVER_PVC_SIZE` with a [config file](/ref/config-files/) or `--set` on `zarf init`. + +To see a full list of `variables` you can view the [zarf.yaml that defines the git-server](https://github.com/defenseunicorns/zarf/blob/main/packages/gitea/zarf.yaml). + +::: + +## Putting it All Together + +The package definition 'init' is similar to writing any other Zarf Package, but with a few key differences: + +Starting with `kind` and `metadata`: + +```yaml {3, 6, 9} +# zarf.yaml +# kind must be ZarfInitConfig +kind: ZarfInitConfig +metadata: + # name *can* be anything, but it is generally recommended to end with 'init' + name: init + # version should be empty as it will be set by the Zarf CLI + # (this is ONLY for the 'init' package) + # version: 0.1.0 +... +``` + +In order for Zarf to operate correctly, the following `components`: + +- must be defined, ordered, and named **exactly** as shown below +- must have the `required` field set to `true` + +```yaml {3, 7, 11, 15} +# zarf.yaml +components: + # components (like k3s) that spin up a cluster... + + - name: zarf-injector + required: true + ... + + - name: zarf-seed-registry + required: true + ... + + - name: zarf-registry + required: true + ... + + - name: zarf-agent + required: true + ... + + # optional components that need a cluster ... +``` + +:::note + +In order to reproduce / build the following example, you will need to have the Zarf repository cloned locally. + +```bash +git clone https://github.com/defenseunicorns/zarf.git +cd zarf +mv zarf.yaml zarf.yaml.bak +``` + +You can learn more about creating a custom init package in the [Creating a Custom 'init' Package Tutorial](/tutorials/8-custom-init-packages). ::: + +A minimal `zarf.yaml` for the 'init' package looks something like: + +```yaml +# zarf.yaml +kind: ZarfInitConfig +metadata: + name: init + +components: + - name: zarf-injector + required: true + import: + path: packages/zarf-registry + + - name: zarf-seed-registry + required: true + import: + path: packages/zarf-registry + + - name: zarf-registry + required: true + import: + path: packages/zarf-registry + + - name: zarf-agent + required: true + import: + path: packages/zarf-agent +``` + +{/* technically the most minimal you can go is just `zarf-agent` and using an external registry / git server but idk if it's worth documenting that */} diff --git a/src/injector/Makefile b/src/injector/Makefile index 8f40fbadbf..a0ba3f467e 100644 --- a/src/injector/Makefile +++ b/src/injector/Makefile @@ -24,8 +24,6 @@ build-injector-linux-amd: ## Build the Zarf injector for AMD64 cargo build --target x86_64-unknown-linux-musl --release; \ fi - du --si target/x86_64-unknown-linux-musl/release/zarf-injector - build-injector-linux-arm: ## Build the Zarf injector for ARM64 rustup target add aarch64-unknown-linux-musl @@ -38,4 +36,10 @@ build-injector-linux-arm: ## Build the Zarf injector for ARM64 cargo build --target aarch64-unknown-linux-musl --release; \ fi +list-sizes: ## List the sizes of the Zarf injector binaries + @echo '\n\033[0;36mSize of Zarf injector binaries:\033[0m\n'; \ + du --si target/x86_64-unknown-linux-musl/release/zarf-injector; \ du --si target/aarch64-unknown-linux-musl/release/zarf-injector + +build-with-docker: ## Build the Zarf injector using Docker + docker run --rm --user "$(id -u)":"$(id -g)" -v $$PWD:/usr/src/zarf-injector -w /usr/src/zarf-injector rust:1.71.0-bookworm make build-injector-linux diff --git a/src/injector/README.md b/src/injector/README.md index 699be652ed..0310ef0467 100644 --- a/src/injector/README.md +++ b/src/injector/README.md @@ -1,20 +1,40 @@ # zarf-injector -A tiny (<1MiB) binary statically-linked with musl in order to fit as a configmap +> If using VSCode w/ the official Rust extension, make sure to open a new window in the `src/injector` directory to make `rust-analyzer` happy. +> +> ```bash +> code src/injector +> ``` -## Building on Ubuntu +A tiny (<1MiB) binary statically-linked with musl in order to fit as a configmap. + +See how it gets used during the [`zarf-init`](https://docs.zarf.dev/commands/zarf_init/) process in the ['init' package reference documentation](https://docs.zarf.dev/ref/init-package/). + +## What does it do? + +```sh +zarf-injector +``` + +The `zarf-injector` binary serves 2 purposes during 'init'. + +1. It re-assembles a multi-part tarball that was split into multiple ConfigMap entries (located at `./zarf-payload-*`) back into `payload.tar.gz`, then extracts it to the `/zarf-seed` directory. It also checks that the SHA256 hash of the re-assembled tarball matches the first (and only) argument provided to the binary. +2. It runs a pull-only, insecure, HTTP OCI compliant registry server on port 5000 that serves the contents of the `/zarf-seed` directory (which is of the OCI layout format). + +This enables a distro-agnostic way to inject real `registry:2` image into a running cluster, thereby enabling air-gapped deployments. + +## Building in Docker (recommended) ```bash -# install rust -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --no-modify-path -source $HOME/.cargo/env +make build-with-docker +``` + +## Building on Debian-based Systems -# install build-essential -sudo apt install build-essential -y +Install [Rust](https://rustup.rs/) and `build-essential`. -# build w/ musl -rustup target add x86_64-unknown-linux-musl -cargo build --target x86_64-unknown-linux-musl --release +```bash +make build-injector-linux list-sizes ``` ## Checking Binary Size @@ -22,12 +42,15 @@ cargo build --target x86_64-unknown-linux-musl --release Due to the ConfigMap size limit (1MiB for binary data), we need to make sure the binary is small enough to fit. ```bash -cargo build --target x86_64-unknown-linux-musl --release +make list-sizes +``` + +```sh +$ make build-with-docker +... -cargo build --target aarch64-unknown-linux-musl --release +Size of Zarf injector binaries: -size_linux=$(du --si target/x86_64-unknown-linux-musl/release/zarf-injector | cut -f1) -echo "Linux binary size: $size_linux" -size_aarch64=$(du --si target/aarch64-unknown-linux-musl/release/zarf-injector | cut -f1) -echo "aarch64 binary size: $size_aarch64" +840k target/x86_64-unknown-linux-musl/release/zarf-injector +713k target/aarch64-unknown-linux-musl/release/zarf-injector ```