Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grafana-rollout-operator: Image with tests #2648

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions generated.tf

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

83 changes: 83 additions & 0 deletions images/grafana-rollout-operator/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
<!--monopod:start-->
# grafana-rollout-operator
| | |
| - | - |
| **OCI Reference** | `cgr.dev/chainguard/grafana-rollout-operator` |


* [View Image in Chainguard Academy](https://edu.chainguard.dev/chainguard/chainguard-images/reference/grafana-rollout-operator/overview/)
* [View Image Catalog](https://console.enforce.dev/images/catalog) for a full list of available tags.
* [Contact Chainguard](https://www.chainguard.dev/chainguard-images) for enterprise support, SLAs, and access to older tags.*

---
<!--monopod:end-->

<!--overview:start-->
Kubernetes Rollout Operator coordinates the rollout of pods between different StatefulSets within a specific namespace, and can be used to manage multi-AZ deployments
<!--overview:end-->

<!--getting:start-->
## Download this Image
The image is available on `cgr.dev`:

```
docker pull cgr.dev/chainguard/grafana-rollout-operator:latest
```
<!--getting:end-->

<!--body:start-->

How to install via Helm only rollout operator:

```bash
helm repo add grafana https://grafana.github.io/helm-charts
kubectl create ns grafana-rollout-operator
helm install grafana-rollout-operator garafana/rollout-operator \
--set image.repository=cgr.dev/chainguard/grafana-rollout-operator \
--set image.tag=latest
```

Get Daemonset count
```bash
kubectl get deploy grafana-rollout-operator -n grafana-rollout-operator
```

This works in conjencture with [mimir-distributed](https://grafana.com/docs/helm-charts/mimir-distributed/latest/get-started-helm-charts/)

Add Labels and Annotations to ingester zones
```bash
kubectl label sts mimir-ingester-zone-a grafana.com/min-time-between-zones-downscale=2m -n mimir
kubectl label sts mimir-ingester-zone-a grafana.com/prepare-downscale=true -n mimir
kubectl annotate sts mimir-ingester-zone-a grafana.com/prepare-downscale-http-path=ingester/prepare-shutdown -n mimir
kubectl annotate sts mimir-ingester-zone-a grafana.com/prepare-downscale-http-port=80 -n mimir

kubectl label sts mimir-ingester-zone-b grafana.com/min-time-between-zones-downscale=2m -n mimir
kubectl label sts mimir-ingester-zone-b grafana.com/prepare-downscale=true -n mimir
kubectl annotate sts mimir-ingester-zone-b grafana.com/rollout-downscale-leader=mimir-ingester-zone-a -n mimir
kubectl annotate sts mimir-ingester-zone-b grafana.com/prepare-downscale-http-path=ingester/prepare-shutdown -n mimir
kubectl annotate sts mimir-ingester-zone-b grafana.com/prepare-downscale-http-port=80 -n mimir

kubectl label sts mimir-ingester-zone-c grafana.com/min-time-between-zones-downscale=2m -n mimir
kubectl label sts mimir-ingester-zone-c grafana.com/prepare-downscale=true -n mimir
kubectl annotate sts mimir-ingester-zone-c grafana.com/rollout-downscale-leader=mimir-ingester-zone-b -n mimir
kubectl annotate sts mimir-ingester-zone-c grafana.com/prepare-downscale-http-path=ingester/prepare-shutdown -n mimir
kubectl annotate sts mimir-ingester-zone-c grafana.com/prepare-downscale-http-port=80 -n mimir
```

Check for logs
```bash
kubectl logs deployment/mimir-rollout-operator -n mimir
```

It should say something like
```bash
level=debug ts=2024-05-14T17:02:34.97299692Z msg="reconciling StatefulSet" statefulset=mimir-store-gateway-zone-a
level=debug ts=2024-05-14T17:02:34.973309878Z msg="reconciling StatefulSet" statefulset=mimir-store-gateway-zone-b
level=debug ts=2024-05-14T17:02:34.97338692Z msg="reconciling StatefulSet" statefulset=mimir-store-gateway-zone-c
level=debug ts=2024-05-14T17:02:34.97363967Z msg="reconciling StatefulSet" statefulset=mimir-ingester-zone-a
level=debug ts=2024-05-14T17:02:34.973721503Z msg="reconciling StatefulSet" statefulset=mimir-ingester-zone-b
level=debug ts=2024-05-14T17:02:34.973819586Z msg="reconciling StatefulSet" statefulset=mimir-ingester-zone-c
```


<!--body:end-->
31 changes: 31 additions & 0 deletions images/grafana-rollout-operator/config/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
terraform {
required_providers {
apko = { source = "chainguard-dev/apko" }
}
}

variable "extra_packages" {
description = "Additional packages to install."
type = list(string)
default = ["grafana-rollout-operator"]
}

module "accts" {
source = "../../../tflib/accts"
run-as = 65532
uid = 65532
gid = 65532
name = "nonroot"
}

output "config" {
value = jsonencode({
contents = {
packages = var.extra_packages
}
accounts = module.accts.block
entrypoint = {
command = "/usr/bin/rollout-operator"
}
})
}
13 changes: 13 additions & 0 deletions images/grafana-rollout-operator/generated.tf

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions images/grafana-rollout-operator/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
terraform {
required_providers {
oci = { source = "chainguard-dev/oci" }
}
}

variable "target_repository" {
description = "The docker repo into which the image and attestations should be published."
}

module "config-latest" { source = "./config" }

module "latest" {
source = "../../tflib/publisher"
name = basename(path.module)
config = module.config-latest.config
target_repository = var.target_repository
build-dev = true
}

module "test-latest" {
source = "./tests"

digest = module.latest.image_ref
}

resource "oci_tag" "latest" {
depends_on = [module.test-latest]
digest_ref = module.latest.image_ref
tag = "latest"
}

resource "oci_tag" "latest-dev" {
depends_on = [module.test-latest]
digest_ref = module.latest.dev_ref
tag = "latest-dev"
}
12 changes: 12 additions & 0 deletions images/grafana-rollout-operator/metadata.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: grafana-rollout-operator
image: cgr.dev/chainguard/grafana-rollout-operator
logo: https://storage.googleapis.com/chainguard-academy/logos/grafana.svg
endoflife: ""
console_summary: ""
short_description: Kubernetes Rollout Operator coordinates the rollout of pods between different StatefulSets within a specific namespace, and can be used to manage multi-AZ deployments
compatibility_notes: ""
readme_file: README.md
upstream_url: https://github.com/grafana/rollout-operator
keywords:
- application
Pyronewbic marked this conversation as resolved.
Show resolved Hide resolved
- kubernetes
69 changes: 69 additions & 0 deletions images/grafana-rollout-operator/tests/logs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#!/usr/bin/env bash

#
# Tests grafana-rollout-operator with its helm chart and reviews the logs
# to ensure expected behaviour
#

set -o errexit -o nounset -o errtrace -o pipefail -x

# Defining log entries we are looking for in the k8s-event-export logs
declare -a terms=(
"informer caches are syncing"
"informer caches have synced"
"reconcile started"
"reconcile done"
)

declare -a missing_terms=()

search_logs() {
local retries=10
local delay=15

for ((i = 1; i <= retries; i++)); do
local logs=$(kubectl logs deployment/mimir-rollout-operator -n mimir 2>&1)
local all_terms_found=true

for term in "${terms[@]}"; do
if echo "$logs" | grep -Fq "$term"; then
echo "Found log term: $term"
else
echo "Log term NOT found: $term (attempt $i of $retries)"
all_terms_found=false
fi
done

if $all_terms_found; then
return 0
elif [[ $i -lt $retries ]]; then
echo "Some log terms were missing. Retrying in $delay seconds..."
sleep $delay

fi
done

# After all retries, record the missing terms
for term in "${terms[@]}"; do
if ! echo "$logs" | grep -Fq "$term"; then
missing_terms+=("$term")
fi
done

echo "FAILED: After $retries attempts, the following terms were not found:"
printf '%s\n' "${missing_terms[@]}"
exit 1
}

TEST_logs_emit_expected_entries() {
# Look for each log term. Will record any which are not found.
search_logs

if [[ ${#missing_terms[@]} -ne 0 ]]; then
echo "The following terms were not found:"
printf '%s\n' "${missing_terms[@]}"
exit 1
fi
}

Pyronewbic marked this conversation as resolved.
Show resolved Hide resolved
TEST_logs_emit_expected_entries
75 changes: 75 additions & 0 deletions images/grafana-rollout-operator/tests/logs_functionality.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/usr/bin/env bash

#
# Tests grafana-rollout-operator with its helm chart and reviews the logs
# to ensure expected behaviour
#

set -o errexit -o nounset -o errtrace -o pipefail -x

# Defining log entries we are looking for in the k8s-event-export logs
declare -a terms=(
"informer caches are syncing"
"informer caches have synced"
"reconcile started"
'"reconciling StatefulSet" statefulset=mimir-store-gateway-zone-a'
'"reconciling StatefulSet" statefulset=mimir-ingester-zone-b'
'"reconciling StatefulSet" statefulset=mimir-ingester-zone-c'
'"reconciling StatefulSet" statefulset=mimir-store-gateway-zone-a'
'"reconciling StatefulSet" statefulset=mimir-store-gateway-zone-b'
'"reconciling StatefulSet" statefulset=mimir-store-gateway-zone-c'
"reconcile done"
)

declare -a missing_terms=()

search_logs() {
local retries=10
local delay=15

for ((i = 1; i <= retries; i++)); do
local logs=$(kubectl logs deployment/mimir-rollout-operator -n mimir 2>&1)
local all_terms_found=true

for term in "${terms[@]}"; do
if echo "$logs" | grep -Fq "$term"; then
echo "Found log term: $term"
else
echo "Log term NOT found: $term (attempt $i of $retries)"
all_terms_found=false
fi
done

if $all_terms_found; then
return 0
elif [[ $i -lt $retries ]]; then
echo "Some log terms were missing. Retrying in $delay seconds..."
sleep $delay

fi
done

# After all retries, record the missing terms
for term in "${terms[@]}"; do
if ! echo "$logs" | grep -Fq "$term"; then
missing_terms+=("$term")
fi
done

echo "FAILED: After $retries attempts, the following terms were not found:"
printf '%s\n' "${missing_terms[@]}"
exit 1
}

TEST_logs_emit_expected_entries() {
# Look for each log term. Will record any which are not found.
search_logs

if [[ ${#missing_terms[@]} -ne 0 ]]; then
echo "The following terms were not found:"
printf '%s\n' "${missing_terms[@]}"
exit 1
fi
}

TEST_logs_emit_expected_entries
Loading
Loading