feat!: update helm chart to support distributed mode and 3.0 (#12067)

Signed-off-by: Edward Welch <edward.welch@grafana.com>
Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>
Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>

docs/sources/setup/upgrade/upgrade-to-6x/index.md

@@ -0,0 +1,89 @@
+---
+title: Upgrade the Helm chart to 6.0
+menuTitle: Upgrade the Helm chart to 6.0
+description: Upgrade the Helm chart from 5.x to 6.0.
+weight: 800
+keywords:
+  - upgrade
+---
+
+## Upgrading to v6.x
+
+v6.x of this chart introduces distributed mode but also introduces breaking changes from v5x.
+
+### Changes
+
+#### BREAKING: `deploymentMode` setting
+
+This only breaks you if you are running the chart in Single Binary mode, you will need to set
+
+```
+deploymentMode: SingleBinary
+```
+
+#### BREAKING: `lokiCanary` section was moved
+
+This section was moved from within the `monitoring` section to the root level of the values file.
+
+#### BREAKING: `topologySpreadConstraints` and `podAffinity` converted to objects
+
+Previously they were strings which were passed through `tpl` now they are normal objects which will be added to deployments.
+
+Also we removed the soft constraint on zone.
+
+#### BREAKING: `externalConfigSecretName` was removed and replaced.
+
+Instead you can now provide `configObjectName` which is used by Loki components for loading the config.
+
+`generatedConfigObjectName` also can be used to control the name of the config object created by the chart.
+
+This gives greater flexibility in using the chart to still generate a config object but allowing for another process to load and mutate this config into a new object which can be loaded by Loki and `configObjectName`
+
+#### Monitoring
+
+After some consideration of how this chart works with other charts provided by Grafana, we decided to deprecate the monitoring sections of this chart and take a new approach entirely to monitoring Loki, Mimir and Tempo with the [Meta Monitoring Chart](https://github.com/grafana/meta-monitoring-chart).
+
+Reasons:
+  * There were conflicts with this chart and the Mimir chart both installing the Agent Operator.
+  * The Agent Operator is deprecated.
+  * The dependency on the Prometheus operator is not one we are able to support well.
+
+The [Meta Monitoring Chart](https://github.com/grafana/meta-monitoring-chart) is an improvement over the the previous approach because it allows for installing a clustered Grafana Agent which can send metrics, logs, and traces to Grafana Cloud, or letting you install a monitoring-only local installation of Loki, Mimir, Tempo, and Grafana.
+
+The monitoring sections of this chart still exist but are disabled by default.
+
+If you wish to continue using the self monitoring features you should use the following configuration, but please do note a future version of this chart will remove this capability completely:
+
+```
+monitoring:
+  enabled: true
+  selfMonitoring:
+    enabled: true
+    grafanaAgent:
+      installOperator: true
+```
+
+#### Memcached is included and enabled by default
+
+Caching is crucial to the proper operation of Loki and Memcached is now included in this chart and enabled by default for the `chunksCache` and `resultsCache`.
+
+If you are already running Memcached separately you can remove your existing installation and use the Memcached deployments built into this chart.
+
+##### Single Binary
+
+Memcached also deploys for the Single Binary, but this may not be desired in resource constrained environments.
+
+You can disable it with the following configuration:
+
+```
+chunksCache:
+  enabled: false
+resultsCache:
+  enabled: false
+```
+
+With these caches disabled, Loki will return to defaults which enables an in-memory results and chunks cache, so you will still get some caching.
+
+#### Distributed mode
+
+This chart introduces the ability to run Loki in distributed, or [microservices mode](https://grafana.com/docs/loki/latest/get-started/deployment-modes/#microservices-mode). Separate instructions on how to enable this as well as how to migrate from the existing community chart will be coming shortly.

production/helm/loki/CHANGELOG.md

@@ -13,6 +13,12 @@ Entries should include a reference to the pull request that introduced the chang
 
 [//]: # (<AUTOMATED_UPDATES_LOCATOR> : do not remove this line. This locator is used by the CI pipeline to automatically create a changelog entry for each new Loki release. Add other chart versions and respective changelog entries bellow this line.)
 
+## 6.0.0
+
+- [CHANGE] the lokiCanary section was moved from under monitoring to be under the root of the file.
+- [CHANGE] the definitions for topologySpreadConstraints and podAffinity were converted from string templates to objects. Also removed the soft constraint on zone.
+- [CHANGE] the externalConfigSecretName was replaced with more generic configs
+
 ## 5.47.2
 
 - [ENHANCEMENT] Allow for additional pipeline stages to be configured on the `selfMonitoring` `Podlogs` resource.

production/helm/loki/Chart.lock

@@ -5,5 +5,8 @@ dependencies:
 - name: grafana-agent-operator
   repository: https://grafana.github.io/helm-charts
   version: 0.3.15
-digest: sha256:b7a42cd0e56544f6168a586fde03e26c801bb20cf69bc004a8f6000d93b98100
-generated: "2024-01-27T21:57:28.190462917+05:30"
+- name: rollout-operator
+  repository: https://grafana.github.io/helm-charts
+  version: 0.13.0
+digest: sha256:d0e60c2879039ee5e8b7b10530f0e8790d6d328ee8afca71f01128627e921587
+generated: "2024-04-07T14:12:43.317329844-04:00"

production/helm/loki/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: loki
 description: Helm chart for Grafana Loki in simple, scalable mode
 type: application
-appVersion: 2.9.6
-version: 5.47.2
+appVersion: 3.0.0
+version: 6.0.0
 home: https://grafana.github.io/helm-charts
 sources:
   - https://github.com/grafana/loki
@@ -21,6 +21,11 @@ dependencies:
     version: 0.3.15
     repository: https://grafana.github.io/helm-charts
     condition: monitoring.selfMonitoring.grafanaAgent.installOperator
+  - name: rollout-operator
+    alias: rollout_operator
+    repository: https://grafana.github.io/helm-charts
+    version: 0.13.0
+    condition: rollout_operator.enabled
 maintainers:
   - name: trevorwhitney
   - name: jeschkies

production/helm/loki/README.md

@@ -1,6 +1,6 @@
 # loki
 
-![Version: 5.47.2](https://img.shields.io/badge/Version-5.47.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.9.6](https://img.shields.io/badge/AppVersion-2.9.6-informational?style=flat-square)
+![Version: 6.0.0](https://img.shields.io/badge/Version-6.0.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 3.0.0](https://img.shields.io/badge/AppVersion-3.0.0-informational?style=flat-square)
 
 Helm chart for Grafana Loki in simple, scalable mode
 
@@ -16,5 +16,6 @@ Helm chart for Grafana Loki in simple, scalable mode
 |------------|------|---------|
 | https://charts.min.io/ | minio(minio) | 4.0.15 |
 | https://grafana.github.io/helm-charts | grafana-agent-operator(grafana-agent-operator) | 0.3.15 |
+| https://grafana.github.io/helm-charts | rollout_operator(rollout-operator) | 0.13.0 |
 
 Find more information in the Loki Helm Chart [documentation](https://grafana.com/docs/loki/next/installation/helm).

production/helm/loki/ci/default-single-binary-values.yaml

@@ -0,0 +1,14 @@
+---
+loki:
+  commonConfig:
+    replication_factor: 1
+  useTestSchema: true
+deploymentMode: SingleBinary
+singleBinary:
+  replicas: 1
+read:
+  replicas: 0
+write:
+  replicas: 0
+backend:
+  replicas: 0

production/helm/loki/ci/default-values.yaml

@@ -2,17 +2,10 @@
 loki:
   commonConfig:
     replication_factor: 1
-  image:
-    tag: "main-5e53303"
+  useTestSchema: true
 read:
   replicas: 1
 write:
   replicas: 1
 backend:
   replicas: 1
-monitoring:
-  serviceMonitor:
-    labels:
-      release: "prometheus"
-test:
-  prometheusAddress: "http://prometheus-kube-prometheus-prometheus.prometheus.svc.cluster.local.:9090"

production/helm/loki/ci/distributed-disabled.yaml

@@ -0,0 +1,32 @@
+---
+loki:
+  commonConfig:
+    replication_factor: 1
+  useTestSchema: true
+deploymentMode: Distributed
+backend:
+  replicas: 0
+read:
+  replicas: 0
+write:
+  replicas: 0
+ingester:
+  replicas: 3 # Kind seems to be a single node for testing so the anti-affinity rules fail here with zone awareness
+querier:
+  replicas: 1
+queryFrontend:
+  replicas: 1
+queryScheduler:
+  replicas: 1
+distributor:
+  replicas: 1
+compactor:
+  replicas: 1
+indexGateway:
+  replicas: 1
+bloomCompactor:
+  replicas: 0
+bloomGateway:
+  replicas: 0
+minio:
+  enabled: true

production/helm/loki/ci/ingress-values.yaml

@@ -11,8 +11,7 @@ gateway:
 loki:
   commonConfig:
     replication_factor: 1
-  image:
-    tag: "main-5e53303"
+  useTestSchema: true
 read:
   replicas: 1
 write:

production/helm/loki/ci/legacy-monitoring-values.yaml

@@ -0,0 +1,22 @@
+---
+loki:
+  commonConfig:
+    replication_factor: 1
+  useTestSchema: true
+read:
+  replicas: 1
+write:
+  replicas: 1
+backend:
+  replicas: 1
+monitoring:
+  enabled: true
+  selfMonitoring:
+    enabled: true
+    grafanaAgent:
+      installOperator: true
+  serviceMonitor:
+    labels:
+      release: "prometheus"
+test:
+  prometheusAddress: "http://prometheus-kube-prometheus-prometheus.prometheus.svc.cluster.local.:9090"

production/helm/loki/distributed-values.yaml

@@ -0,0 +1,70 @@
+---
+loki:
+  schemaConfig:
+    configs:
+      - from: 2024-04-01
+        store: tsdb
+        object_store: s3
+        schema: v13
+        index:
+          prefix: loki_index_
+          period: 24h
+  ingester:
+    chunk_encoding: snappy
+  tracing:
+    enabled: true
+  querier:
+    # Default is 4, if you have enough memory and CPU you can increase, reduce if OOMing
+    max_concurrent: 4
+
+#gateway:
+#  ingress:
+#    enabled: true
+#    hosts:
+#      - host: FIXME
+#        paths:
+#          - path: /
+#            pathType: Prefix
+
+deploymentMode: Distributed
+
+ingester:
+  replicas: 3
+querier:
+  replicas: 3
+  maxUnavailable: 2
+queryFrontend:
+  replicas: 2
+  maxUnavailable: 1
+queryScheduler:
+  replicas: 2
+distributor:
+  replicas: 3
+  maxUnavailable: 2
+compactor:
+  replicas: 1
+indexGateway:
+  replicas: 2
+  maxUnavailable: 1
+
+bloomCompactor:
+  replicas: 0
+bloomGateway:
+  replicas: 0
+
+# Enable minio for storage
+minio:
+  enabled: true
+
+# Zero out replica counts of other deployment modes
+backend:
+  replicas: 0
+read:
+  replicas: 0
+write:
+  replicas: 0
+
+singleBinary:
+  replicas: 0
+
+

production/helm/loki/simple-scalable-values.yaml

@@ -0,0 +1,63 @@
+---
+loki:
+  schemaConfig:
+    configs:
+      - from: 2024-04-01
+        store: tsdb
+        object_store: s3
+        schema: v13
+        index:
+          prefix: loki_index_
+          period: 24h
+  ingester:
+    chunk_encoding: snappy
+  tracing:
+    enabled: true
+  querier:
+    # Default is 4, if you have enough memory and CPU you can increase, reduce if OOMing
+    max_concurrent: 4
+
+#gateway:
+#  ingress:
+#    enabled: true
+#    hosts:
+#      - host: FIXME
+#        paths:
+#          - path: /
+#            pathType: Prefix
+
+deploymentMode: SimpleScalable
+
+backend:
+  replicas: 3
+read:
+  replicas: 3
+write:
+  replicas: 3
+
+# Enable minio for storage
+minio:
+  enabled: true
+
+# Zero out replica counts of other deployment modes
+singleBinary:
+  replicas: 0
+
+ingester:
+  replicas: 0
+querier:
+  replicas: 0
+queryFrontend:
+  replicas: 0
+queryScheduler:
+  replicas: 0
+distributor:
+  replicas: 0
+compactor:
+  replicas: 0
+indexGateway:
+  replicas: 0
+bloomCompactor:
+  replicas: 0
+bloomGateway:
+  replicas: 0