Helm chart v1.0.0

README.md

@@ -46,6 +46,7 @@ At present, control plane logs are not covered as part of out of the box collect
 The out of the box collection for these logs will be available soon for generic Kubernetes clusters and for OKE (when OKE makes these logs accessible to end users).
 
 ### Application Pod/Container Logs
+
 All the logs from application pods writing STDOUT/STDERR are typically available under /var/log/containers/. 
 Application which are having custom log handlers (say log4j or similar) may route their logs differently but in general would be available on the node (through a volume).
 
@@ -91,7 +92,7 @@ The following are the list of objects supported at present:
 We are in the process of building a docker image based off Oracle Linux 8 including Fluentd, OCI Logging Analytics Output Plugin and all the required dependencies. 
 All the dependencies will be build from source and installed into the image. This image soon would be available to use as a pre-built image as is (OR) to create a custom image using this image as a base image.
 At present, for testing purposes follow the below mentioned steps to build an image using official Fluentd Docker Image as base image (off Debian).
-- Download all the files from [this dir](/logan/docker-images/v1.0/debian/) into a local machine having access to internet.
+- Download all the files from [this dir](logan/docker-images/v1.0/debian/) into a local machine having access to internet.
 - Run the following command to build the docker image.
     - *docker build -t fluentd_oci_la -f Dockerfile .*
 - The docker image built from the above step, can either be pushed to Docker Hub or OCI Container Registry (OCIR) or to a Local Docker Registry depending on the requirements.
@@ -107,7 +108,7 @@ At present, for testing purposes follow the below mentioned steps to build an im
 
 #### To enable Logs collection
 
-Download all the yaml files from [this dir](/logan/kubernetes-resources/logs-collection/).
+Download all the yaml files from [this dir](logan/kubernetes-resources/logs-collection/).
 These yaml files needs to be applied using kubectl to create the necessary resources that enables the logs collection into Logging Analytics through a Fluentd based DaemonSet.
 
 ##### configmap-docker.yaml | configmap-cri.yaml
@@ -163,7 +164,7 @@ kubectl rollout restart daemonset oci-la-fluentd-daemonset -n=kube-system
 
 #### To enable Kubernetes Objects collection
 
-Download all the yaml files from [this dir](/logan/kubernetes-resources/objects-collection/).
+Download all the yaml files from [this dir](logan/kubernetes-resources/objects-collection/).
 These yaml files needs to be applied using kubectl to create the necessary resources that enables the Kuberetes Objects collection into Logging Analytics.
 
 ##### configMap-objects.yaml
@@ -202,15 +203,42 @@ kubectl rollout restart deployment oci-la-fluentd-deployment -n=kube-system
 
 ### Deploying Kuberenetes resources using Helm
 
-Coming soon ...
+#### Pre-requisites
+
+- Install helm if not done already. Refer [this](https://helm.sh/docs/intro/install/).
+- Download the helm chart from [this dir](logan/helm-chart/).
 
+#### values.yaml
 
+- This file contains all the default values possible to setup the logs and objects collection, but few values needs to be provided either through an external values.yaml file or by modifying this file. It is recommended to use external values.yaml to override any values.
+- Inline documentation has the description and possible values for each of the configuration parameters.
+- At minimum, the following needs to be set accordingly. image:url, ociLANamespace, ociLALogGroupID. It is recommended to set kubernetesClusterID and kubernetesClusterName too, to tag all the logs processed with corresponding Kubernetes cluster at Logging Analytics. 
+- Use "docker" as runtime for Kubernetes clusters based off Docker runtime (e.g., OKE < 1.20) and "cri" for Kubernetes clusters based off CRI-O. The default is "cri".
+- Use "InstancePrincipal" as authtype for OKE and all clusters which are running on OCI VMs and "config" as authtype for OCI Config file based Auth/AuthZ. config under oci section needs to be updated with relevant info when authtype is chosen as "config". The default is "InstancePrincipal".
 
+#### Commands Reference
 
+It is recommended to validate the values using the following `helm template` command before actually installing. Provide path to exterval values.yaml and path to helm-chart.
+
+```
+helm template --values <path-to-external-values.yaml> <path-to-helm-chart>
+```
 
+Now, the chart can be installed using the following `helm install` command. Provide a desired release name, path to exterval values.yaml and path to helm-chart.
 
+```
+helm install <release-name> --values <path-to-external-values.yaml> <path-to-helm-chart>
+```
 
+Use the following `helm upgrade` command if any further changes to values.yaml needs to be applied or a new chart version needs to be deployed. Refer [this](https://helm.sh/docs/helm/helm_upgrade/) for further details on `helm upgrade`.
 
+```
+helm upgrade <release-name> --values <path-to-external-values.yaml> <path-to-helm-chart>
+```
 
+Use the following `helm uninstall` command to delete the chart. Provide the release name used when creating the chart.
 
-
+```
+helm uninstall <release-name>
+```
+

logan/helm-chart/Chart.yaml

@@ -0,0 +1,25 @@
+apiVersion: v2
+name: oci-la-fluentd
+description: Helm chart for collecting Kubernetes logs and objects using Fluentd into OCI Logging Analytics.
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 1.0.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+# appVersion here reflects OCI Logging Analytics Fluentd Output Plugin version
+appVersion: "2.0.0"

logan/helm-chart/templates/_helpers.tpl

@@ -0,0 +1,14 @@
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "oci-la-fluentd.name" -}}
+{{- default .Chart.Name .Values.name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{- define "common.tplvalues.render" -}}
+    {{- if typeIs "string" .value }}
+        {{- tpl .value .context }}
+    {{- else }}
+        {{- tpl (.value | toYaml) .context }}
+    {{- end }}
+{{- end -}}

logan/helm-chart/templates/clusterrole-logs.yaml

@@ -0,0 +1,15 @@
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: {{ include "oci-la-fluentd.name" . }}-logs-clusterrole
+  namespace: {{ default "kube-system" .Values.namespace }}
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - '*'
+  verbs:
+  - get
+  - list
+  - watch

logan/helm-chart/templates/clusterrole-objects.yaml

@@ -0,0 +1,23 @@
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: {{ include "oci-la-fluentd.name" . }}-objects-clusterrole
+  namespace: {{ default "kube-system" .Values.namespace }}
+rules:
+  - apiGroups:
+      - ""
+    resources:
+      - '*'
+    verbs:
+      - get
+      - list
+      - watch
+  - apiGroups:
+      - apps
+    resources:
+      - '*'
+    verbs:
+      - get
+      - list
+      - watch

logan/helm-chart/templates/clusterrolebinding-logs.yaml

@@ -0,0 +1,13 @@
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: {{ include "oci-la-fluentd.name" . }}-logs-clusterrolebinding
+roleRef:
+  kind: ClusterRole
+  name: {{ include "oci-la-fluentd.name" . }}-logs-clusterrole
+  apiGroup: rbac.authorization.k8s.io
+subjects:
+- kind: ServiceAccount
+  name: {{ include "oci-la-fluentd.name" . }}-serviceaccount
+  namespace: {{ default "kube-system" .Values.namespace }}

logan/helm-chart/templates/clusterrolebinding-objects.yaml

@@ -0,0 +1,13 @@
+---
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: {{ include "oci-la-fluentd.name" . }}-objects-clusterrolebinding
+roleRef:
+  kind: ClusterRole
+  name: {{ include "oci-la-fluentd.name" . }}-objects-clusterrole
+  apiGroup: rbac.authorization.k8s.io
+subjects:
+- kind: ServiceAccount
+  name: {{ include "oci-la-fluentd.name" . }}-serviceaccount
+  namespace: {{ default "kube-system" .Values.namespace }}

logan/helm-chart/templates/configmap-logs.yaml

@@ -0,0 +1,11 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ include "oci-la-fluentd.name" . }}-logs-configmap
+  namespace: {{ default "kube-system" .Values.namespace }}
+data:
+  # file-like keys
+  {{- range $key, $value := .Values.fluentd.configMapLogsFiles }}
+  {{ $key }}: |
+    {{- include "common.tplvalues.render" (dict "value" $value "context" $) | nindent 4 }}
+  {{- end }}

logan/helm-chart/templates/configmap-objects.yaml

@@ -0,0 +1,11 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ include "oci-la-fluentd.name" . }}-objects-configmap
+  namespace: {{ default "kube-system" .Values.namespace }}
+data:
+  # file-like keys
+  {{- range $key, $value := .Values.fluentd.configMapObjectsFiles }}
+  {{ $key }}: |
+    {{- include "common.tplvalues.render" (dict "value" $value "context" $) | nindent 4 }}
+  {{- end }}

logan/helm-chart/templates/fluentd-daemonset.yaml

@@ -0,0 +1,115 @@
+---
+{{- $authtype := .Values.authtype | lower }}
+{{- $imagePullSecrets := .Values.image.imagePullSecrets }}
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+  name: {{ include "oci-la-fluentd.name" . }}-daemonset
+  namespace: {{ default "kube-system" .Values.namespace }}
+  labels:
+    app: {{ include "oci-la-fluentd.name" . }}-logs
+    version: v1
+spec:
+  selector:
+    matchLabels:
+      app: {{ include "oci-la-fluentd.name" . }}-logs
+      version: v1
+  template:
+    metadata:
+      annotations:
+        {{- if eq $authtype "config" }}
+        checksum/secrets: {{ include (print $.Template.BasePath "/oci-config-secrets.yaml") . | sha256sum }}
+        {{- end}}
+        checksum/configmap: {{ include (print $.Template.BasePath "/configmap-logs.yaml") . | sha256sum }}
+      labels:
+        app: {{ include "oci-la-fluentd.name" . }}-logs
+        version: v1
+    spec:
+      serviceAccountName: {{ include "oci-la-fluentd.name" . }}-serviceaccount
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        effect: NoSchedule
+      {{- if $imagePullSecrets }}	
+      imagePullSecrets:		  
+      - name: {{ .Values.image.imagePullSecrets }}
+      {{- end}}
+      containers:
+      - name: {{ include "oci-la-fluentd.name" . }}-logs
+        image: {{ .Values.image.url }}
+        imagePullPolicy: {{ default "IfNotPresent" .Values.image.imagePullPolicy }}
+        env:
+          - name: FLUENTD_CONF
+            value: {{ .Values.fluentd.path }}/{{ .Values.fluentd.file }}
+          - name: K8S_NODE_NAME
+            valueFrom:
+              fieldRef:
+                fieldPath: spec.nodeName
+          - name: FLUENT_OCI_DEFAULT_LOGGROUP_ID
+            value: {{ .Values.ociLALogGroupID }}
+          - name: FLUENT_OCI_NAMESPACE
+            value: {{ .Values.ociLANamespace }}
+          - name: FLUENT_OCI_KUBERNETES_CLUSTER_ID
+            value: {{ .Values.kubernetesClusterID }}
+          - name: FLUENT_OCI_KUBERNETES_CLUSTER_NAME
+            value: {{ .Values.kubernetesClusterName }}
+          {{- if eq $authtype "config" }}
+          - name: FLUENT_OCI_CONFIG_LOCATION
+            value: {{ .Values.oci.path }}/{{ .Values.oci.file }}
+          {{- end }} 
+          {{- if .Values.extraEnv }}
+          {{- toYaml .Values.extraEnv | nindent 10 }}
+          {{- end }}
+        {{- if .Values.resources }}
+        resources: {{- toYaml .Values.resources | nindent 10 }}
+        {{- end }}
+        volumeMounts:
+        - name: varlog
+          mountPath: /var/log
+        # Mount all relevant locations depending on where the actual logs presents.
+        - name: dockercontainerlogdirectory
+          mountPath: {{ .Values.volumes.podsHostPath }}
+          readOnly: true
+        - name: dockercontainerdatadirectory
+          mountPath: {{ .Values.volumes.containerdataHostPath }}
+          readOnly: true  
+        {{- if eq $authtype "config" }}
+        # Mount directory where oci config exists
+        - name: ociconfigdir
+          mountPath: {{ .Values.oci.path }}
+          readOnly: true
+        {{- end }}
+        # Mount directory where fluentd config exists
+        - name: fluentdconfigdir
+          mountPath: {{ .Values.fluentd.path }}
+          readOnly: true
+        {{- if .Values.extraVolumeMounts }}
+        {{- toYaml .Values.extraVolumeMounts | nindent 8 }}
+        {{- end }}
+      terminationGracePeriodSeconds: 30
+      volumes:
+      {{- if .Values.extraVolumes }}
+      {{- toYaml .Values.extraVolumes | nindent 6 }}
+      {{- end }}
+      - name: varlog
+        hostPath:
+          path: /var/log
+      # Mount all relevant locations depending on where the actual logs presents.
+      - name: dockercontainerlogdirectory
+        hostPath:
+          path: {{ .Values.volumes.podsHostPath }}
+      - name: dockercontainerdatadirectory
+        hostPath:
+          path: {{ .Values.volumes.containerdataHostPath }}		  
+      {{- if eq $authtype "config" }}
+      # Mount directory where oci config exists
+      - name: ociconfigdir
+        projected:
+          sources:
+              - secret:
+                  name: {{ include "oci-la-fluentd.name" . }}-credentials-secret
+      {{- end }}
+      # Mount directory where fluentd config exists
+      - name: fluentdconfigdir
+        configMap:
+          # Provide the name of the ConfigMap to mount.
+          name: {{ include "oci-la-fluentd.name" . }}-logs-configmap