Skip to content

Commit

Permalink
Merge branch 'main' into ez/add-tool-versions
Browse files Browse the repository at this point in the history
  • Loading branch information
Racer159 committed Mar 4, 2024
2 parents 1ea4782 + 8718e58 commit b7ddc83
Show file tree
Hide file tree
Showing 74 changed files with 998 additions and 876 deletions.
40 changes: 20 additions & 20 deletions CODEOWNERS
Validating CODEOWNERS rules …
@@ -1,35 +1,35 @@
* @defenseunicorns/zarf

# Docs & examples
/adr/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/docs/ @Madeline-UX @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/examples/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16
*.md @Madeline-UX @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/adr/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/docs/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/examples/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
*.md @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321

# Core code
/src/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/go.* @jeff-mccoy @Racer159 @Noxsios @lucasrod16
main.go @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/src/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/go.* @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
main.go @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321

# Init package
/packages/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/zarf.yaml @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/packages/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/zarf.yaml @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321

# Docs Website
/docs-website/ @mike-winberry @Racer159 @Noxsios @jeff-mccoy @lucasrod16
/docs-website/ @Racer159 @Noxsios @jeff-mccoy @lucasrod16 @AustinAbro321

# Privileged pipeline files
/.github/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/hack/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/.gitignore @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/.golangci.yml @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/.goreleaser.yml @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/.grype.yaml @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/Dockerfile @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/renovate.json @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/Makefile @jeff-mccoy @Racer159 @Noxsios @lucasrod16
/.github/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/hack/ @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/.gitignore @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/.golangci.yml @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/.goreleaser.yml @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/.grype.yaml @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/Dockerfile @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/renovate.json @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321
/Makefile @jeff-mccoy @Racer159 @Noxsios @lucasrod16 @AustinAbro321

# Additional privileged files
/CODEOWNERS @jeff-mccoy @mikevanhemert @austenbryan
/CODEOWNERS @jeff-mccoy @austenbryan
/cosign.pub @jeff-mccoy @austenbryan
/LICENSE @jeff-mccoy @austenbryan
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -62,6 +62,7 @@ _<https://www.youtube.com/watch?v=WnOYlFVVKDE>_
To try Zarf out for yourself, visit the ["Try It Now"](https://zarf.dev/install) section on our website.

To learn more about Zarf and its use cases, visit [docs.zarf.dev](https://docs.zarf.dev/docs/zarf-overview). From the docs, you can learn more about:

- [installation](https://docs.zarf.dev/docs/getting-started/#installing-zarf)
- [using the CLI](https://docs.zarf.dev/docs/the-zarf-cli/),
- [making packages](https://docs.zarf.dev/docs/create-a-zarf-package/zarf-packages/),
Expand Down Expand Up @@ -93,7 +94,6 @@ To dive deeper into the tech, you can read the [Nerd Notes](https://docs.zarf.de

[Source DrawIO](docs/.images/architecture.drawio.svg)


## ⭐️ Special Thanks

> Early Zarf research and prototypes were developed jointly with [United States Naval Postgraduate School](https://nps.edu/) research you can read [here](https://calhoun.nps.edu/handle/10945/68688).
Expand Down
4 changes: 2 additions & 2 deletions adr/0012-local-image-support-via-docker.md
Expand Up @@ -8,7 +8,7 @@ Accepted

## Context

There has been a long-standing usabilty gap with Zarf when doing local development due to a lack of local image support. A solution was merged in, [#1173](https://github.com/defenseunicorns/zarf/pull/1173), and released in [v0.23.4](https://github.com/defenseunicorns/zarf/releases/tag/v0.23.4) to support this feature. Unfortuantely, we didn't realize there is a [glaring issue](https://github.com/defenseunicorns/zarf/issues/1214) with the implementation that causes Zarf to crash when trying to load large images into the local docker daemon. The docker daemon support in Crane is somewhat naive and can send a machine into an OOM condition due to how the tar stream is loaded into memory from the docker save action. Crane does have an option to avoid this issue, but at the cost of being much slower to load images from docker.
There has been a long-standing usability gap with Zarf when doing local development due to a lack of local image support. A solution was merged in, [#1173](https://github.com/defenseunicorns/zarf/pull/1173), and released in [v0.23.4](https://github.com/defenseunicorns/zarf/releases/tag/v0.23.4) to support this feature. Unfortunately, we didn't realize there is a [glaring issue](https://github.com/defenseunicorns/zarf/issues/1214) with the implementation that causes Zarf to crash when trying to load large images into the local docker daemon. The docker daemon support in Crane is somewhat naive and can send a machine into an OOM condition due to how the tar stream is loaded into memory from the docker save action. Crane does have an option to avoid this issue, but at the cost of being much slower to load images from docker.

We did extensive investigation into various strategies of loading docker images from the daemon including: crane, skopeo, the docker go client and executing the docker cli directly with varying levels of success. Unfortunately, some of the methods that did work well were up to 3 times slower than the current implementation, though they avoided the OOM issue. Lastly, the docker daemon save operations directly still ended up being slower than crane and docker produced a legacy format that would cause issues with [future package schema changes](https://github.com/defenseunicorns/zarf/issues/1319) we are planning for oci imports.

Expand All @@ -21,7 +21,7 @@ We did extensive investigation into various strategies of loading docker images

## Decision

We had hoped to leverage docker caching and avoid crane caching moreso, but realized that caching was still occuring via Syft for SBOM. Additionaly, the extremely-large, local-only image is actually the edge case here and we created a recommended workaround in the FAQs as well as an inline alert when a large docker image is detected. This restores behavior to what it was before the docker daemon support was added, but with the added benefit of being able to load images from the docker daemon when they are available locally.
We had hoped to leverage docker caching and avoid crane caching moreso, but realized that caching was still occurring via Syft for SBOM. Additionally, the extremely-large, local-only image is actually the edge case here and we created a recommended workaround in the FAQs as well as an inline alert when a large docker image is detected. This restores behavior to what it was before the docker daemon support was added, but with the added benefit of being able to load images from the docker daemon when they are available locally.

## Consequences

Expand Down
20 changes: 10 additions & 10 deletions adr/0018-hooks.md
Expand Up @@ -47,7 +47,7 @@ Potential considerations:
```

Example Secret Data:

```yaml
hookName: ecr-repository
internal: true
Expand All @@ -59,25 +59,25 @@ Potential considerations:
```

For this solution, hooks have to be 'installed' onto a cluster before they are used. When Zarf is deploying a package onto a cluster, it will look for any secrets with the `zarf-hook` label in the `zarf` namespace. If hooks are found, Zarf will run any 'package' level hooks before deploying a component and run any 'component' level hook for each component that is getting deployed. The hook lifecycle options will be:

1. Before a package deployment
2. After a package deployment
3. Before a component deployment
4. After a component deployment

NOTE: The order of hook execution is nearly random. If there are multiple hooks for a lifecycle there is no guarantee that they will be executed in a certain order.
NOTE: The `package` lifecycle might be changed to a `run-once` lifecycle. This would benefit packages that don't have kube context information when the deployment starts.

Zarf hooks will have two forms of execution via `Internal` and `External` hooks:

Internal Hooks:

Internal hooks will be hooks that are built into the Zarf CLI and run internal code when executed. The logic for these hooks would be built into the Zarf CLI and would be updated with new releases of the CLI.

External Hooks:

There are a few approaches for external hooks:

1. Have the hook metadata reference an OCI image that is downloaded and run.

- The hook metadata can reference the shasum of the image to ensure the image is not tampered with.
Expand All @@ -88,12 +88,12 @@ Potential considerations:

1. Have the hook metadata contain a script or list of shell commands that can get run.
- This would be the simplest solution but would require the most work from the hook creator. This also has the most potential security issues.

Pros:

- Implementing Hooks internally means we don't have to deal with any bootstrapping issues.
- Internally managed hooks can leverage Zarf internal code.

Cons:

- Since 'Internal' hooks are built into the CLI, the only way to get updates for the hook is to update the CLI.
Expand Down
9 changes: 6 additions & 3 deletions adr/0019-decouple-transformation-library-from-zarf.md
Expand Up @@ -8,9 +8,10 @@ Pending

## Context

We need a way to decouple transformations from Zarf. We know the Zarf transformation library is battle tested, hardened, and reliable. As Pepr takes admission responsibilities from the Zarf Agent, we need a way to synchronize the TypeScript with the Go code instead of maintaining two disparate libraries which will be expected to grow.
We need a way to decouple transformations from Zarf. We know the Zarf transformation library is battle tested, hardened, and reliable. As Pepr takes admission responsibilities from the Zarf Agent, we need a way to synchronize the TypeScript with the Go code instead of maintaining two disparate libraries which will be expected to grow.

We considered:

- WASM
- gRPC
- REST
Expand All @@ -19,6 +20,7 @@ We considered:
#### WASM

**PROS**

- Shared codebase between TypeScript and Go
- Fast
- API Contract
Expand All @@ -30,6 +32,7 @@ We considered:
#### gRPC

**PROS**

- Shared codebase between TypeScript and Go
- Fast
- API Contract
Expand All @@ -40,6 +43,7 @@ We considered:
#### REST

**PROS**

- Shared codebase between TypeScript and Go
- Proven
- API Contract
Expand All @@ -50,13 +54,12 @@ We considered:
#### Rewrite in TypeScript

**PROS**

- Low hanging fruit
**CONS**
- Two codebases to maintain
- TypeScript is not as battle tested as Go



## Decision

We conformed around using WASM because we can leverage the battle tested transform library from Zarf without incurring network overhead cost from pod to pod or container to container communications.
Expand Down
6 changes: 4 additions & 2 deletions docs-website/static-components/README.md
@@ -1,6 +1,8 @@
# Static Components
## Why?

## Why?

- Docusaurus only allows stringified html to be injected into the theme based components.
- This allows for more robust customization without re-writing things like the navbar or the footer.
- Also allows for cleaner writing of css and html with interpolation and re-use of components.
- see SocialsBox for example.
- see SocialsBox for example.
1 change: 0 additions & 1 deletion docs/1-getting-started/index.md
Expand Up @@ -99,7 +99,6 @@ This example shows how to install Zarf with the official (📜) `defenseunicorns

## Windows Commands


:::note

There is currently no Zarf quick start for Windows, though you can learn how to install Zarf from our Github Releases by visiting the [Installing Zarf page](./0-installing-zarf.md#downloading-the-cli-from-github-releases).
Expand Down
1 change: 0 additions & 1 deletion docs/2-the-zarf-cli/0-building-your-own-cli.md
Expand Up @@ -16,7 +16,6 @@ If you are running `make` targets other than the `build-cli-*` targets described

:::


If you don't want to build the CLI yourself, you can download a pre-compiled binary from the 'Assets' tab of our latest [releases](https://github.com/defenseunicorns/zarf/releases) on GitHub. The pre-compiled binary is available for both arm64 and amd64 machines.

## Building the CLI
Expand Down
2 changes: 1 addition & 1 deletion docs/2-the-zarf-cli/2-zarf-config-files.md
Expand Up @@ -42,4 +42,4 @@ Zarf searches for the Zarf Config File from either your current working director
<TabItem value="json">
<FetchFileCodeBlock src={require('../../examples/config-file/zarf-config.json')} fileFormat="json" fileName="zarf-config.json" />
</TabItem>
</Tabs>
</Tabs>
4 changes: 2 additions & 2 deletions docs/5-zarf-tutorials/0-creating-a-zarf-package.md
Expand Up @@ -14,8 +14,8 @@ When creating a Zarf package, you must have a network connection so that Zarf ca

Before beginning this tutorial you will need the following:

- Zarf binary installed on your $PATH: ([Installing Zarf](../1-getting-started/index.md#installing-zarf))
- A text editor or development environment such as [VS Code](../3-create-a-zarf-package/8-vscode.md)
- Zarf binary installed on your $PATH: ([Installing Zarf](../1-getting-started/index.md#installing-zarf))
- A text editor or development environment such as [VS Code](../3-create-a-zarf-package/8-vscode.md)

## Putting Together a Zarf Package

Expand Down
5 changes: 3 additions & 2 deletions docs/5-zarf-tutorials/10-package-create-differential.md
Expand Up @@ -17,9 +17,11 @@ For following along locally, please ensure the following prerequisites are met:
- The [Zarf](https://github.com/defenseunicorns/zarf) repository cloned: ([`git clone` Instructions](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository))

## Youtube Tutorial

[![Tutorial: Create Differential Packages with Zarf](../.images/tutorials/differential_package_thumbnail.png)](https://youtu.be/6yQEo4Vhz9o "Create Differential Packages with Zarf")

## Create a Differential Package

In this example we're going to use the [Longhorn Example](../../examples/longhorn/) to create a differential package.

:::note
Expand All @@ -36,7 +38,7 @@ You'll notice we're not using a cluster in this example. That's because differe

3. Update the version of the package as well in the [metadata.version](/docs/create-a-zarf-package/zarf-schema#metadata) field.

:::caution
:::caution

Creating differential packages without updating the package version will result in an error.

Expand All @@ -55,4 +57,3 @@ Creating differential packages without updating the package version will result
You have now learned to create differential packages with Zarf. If you would like to see the size you saved, you can use the `zarf package create` command without the `--differential` flag. You'll notice a size difference between the two packages since the differential package omits duplicate contents.

<iframe src="/docs/tutorials/differential_package_create_conclusion.html" width="100%" height="600px"></iframe>

5 changes: 2 additions & 3 deletions docs/8-faq.md
Expand Up @@ -6,7 +6,7 @@ sidebar_position: 8

## Who is behind this project?

Zarf was built by the developers at [Defense Unicorns](https://www.defenseunicorns.com/) and an amazing community of contributors.
Zarf was built by the developers at [Defense Unicorns](https://www.defenseunicorns.com/) and an amazing community of contributors.

Defense Unicorns' mission is to advance freedom and independence globally through Free and Open Source software.

Expand All @@ -16,7 +16,7 @@ Zarf is under the [Apache License 2.0](https://github.com/defenseunicorns/zarf/b

## Is Zarf free to use?

Yes! Zarf is Free and Open-Source Software (FOSS). And will remain free forever. We believe Free and Open Source software changes the world and promotes freedom and security. Anyone who sees the value in our tool should be free to use it without fear of vendor locking or licensing fees.
Yes! Zarf is Free and Open-Source Software (FOSS). And will remain free forever. We believe Free and Open Source software changes the world and promotes freedom and security. Anyone who sees the value in our tool should be free to use it without fear of vendor locking or licensing fees.

## Do I have to use Homebrew to install Zarf?

Expand All @@ -26,7 +26,6 @@ No, the Zarf binary and init package can be downloaded from the [Releases Page](

Zarf is statically compiled and written in [Go](https://golang.org/) and [Rust](https://www.rust-lang.org/), so it has no external dependencies. For Linux, Zarf can bring a Kubernetes cluster using [K3s](https://k3s.io/). For Mac and Windows, Zarf can leverage any available local or remote cluster the user has access to. Currently, the K3s installation Zarf performs does require a [Systemd](https://en.wikipedia.org/wiki/Systemd) based system and `root` (not just `sudo`) access.


## What is the Zarf Agent?

The Zarf Agent is a [Kubernetes Mutating Webhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook) that is installed into the cluster during `zarf init`. The Agent is responsible for modifying [Kubernetes PodSpec](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec) objects [Image](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Container.Image) fields to point to the Zarf Registry. This allows the cluster to pull images from the Zarf Registry instead of the internet without having to modify the original image references. The Agent also modifies [Flux GitRepository](https://fluxcd.io/docs/components/source/gitrepositories/) objects to point to the local Git Server.
Expand Down
18 changes: 9 additions & 9 deletions docs/9-roadmap.md
Expand Up @@ -47,25 +47,25 @@ Zarf has three levels of feature stability that are defined below but generally

Alpha features act as early previews for things that are being considered for inclusion into Zarf and:

- they are very likely to have breaking changes (or be removed entirely) in future versions of Zarf
- they will have limited documentation beyond autogenerated CLI and schema documentation (questions should be directed to the [Zarf team](https://kubernetes.slack.com/archives/C03B6BJAUJ3))
- they will have limited testing and are only recommended for use cases where the risk of instability is acceptable
- they are very likely to have breaking changes (or be removed entirely) in future versions of Zarf
- they will have limited documentation beyond autogenerated CLI and schema documentation (questions should be directed to the [Zarf team](https://kubernetes.slack.com/archives/C03B6BJAUJ3))
- they will have limited testing and are only recommended for use cases where the risk of instability is acceptable

### `beta`

Beta features are newer features that are being prepared to eventually become stable features and:

- they may introduce breaking changes before release, but will include any applicable workarounds
- they will have documentation beyond autogenerated docs, but it won't be fully complete
- they will have tests that cover their core features, but not covering every flavor of environment
- they may introduce breaking changes before release, but will include any applicable workarounds
- they will have documentation beyond autogenerated docs, but it won't be fully complete
- they will have tests that cover their core features, but not covering every flavor of environment

### `stable`

Stable features are features that the Zarf team has committed to maintaining stability for and:

- they will have backwards compatibility shims introduced in lieu of breaking changes
- they will have autogenerated docs, tutorials/tutorial mentions, examples, and core docs
- they will have tests that cover the feature which will run across all tested environments
- they will have backwards compatibility shims introduced in lieu of breaking changes
- they will have autogenerated docs, tutorials/tutorial mentions, examples, and core docs
- they will have tests that cover the feature which will run across all tested environments

:::note

Expand Down
4 changes: 4 additions & 0 deletions examples/argocd/baseline/values.yaml
Expand Up @@ -14,3 +14,7 @@ configs:
url: https://github.com/stefanprodan/podinfo.git
username: ""
password: ""

redis:
image:
repository: docker.io/library/redis
2 changes: 1 addition & 1 deletion examples/argocd/zarf.yaml
Expand Up @@ -15,7 +15,7 @@ components:
valuesFiles:
- baseline/values.yaml
images:
- public.ecr.aws/docker/library/redis:7.0.15-alpine
- docker.io/library/redis:7.0.15-alpine
- quay.io/argoproj/argocd:v2.9.6
# Cosign artifacts for images - argocd - argocd-helm-chart
- quay.io/argoproj/argocd:sha256-2dafd800fb617ba5b16ae429e388ca140f66f88171463d23d158b372bb2fae08.sig
Expand Down

0 comments on commit b7ddc83

Please sign in to comment.