helm-201.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Helm 201\n",
    "\n",
    "A deep-dive into Helm (v3) and details like\n",
    "\n",
    "* Templating\n",
    "* Charts and Subcharts\n",
    "* Usage and internal structure in Kubernetes\n",
    "* Integrations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wd_init = \"work/helm-init2\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!helm version\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "\n",
    "## Init\n",
    "\n",
    "* Create a new template / Helm Chart"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo $wd_init\n",
    "!mkdir -p $wd_init\n",
    "\n",
    "!helm create $wd_init/demo-helm-201\n",
    "\n",
    "!tree $wd_init"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chart Structure and Overview\n",
    "\n",
    "* `Chart.yaml` common meta information\n",
    "  * avoid using `appVersion` - handle version in env-specific value-files or as field which will be overwritten during helm execution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cat $wd_init/demo-helm-201/Chart.yaml | grep -B2 -i 'version:'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "* Templating\n",
    "  * include output/result from namespaced functions\n",
    "  * Whitespaces and new lines, indent\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Action (00)\n",
    "  * modify service template"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Render template and generate Kubernetes resource files\"\n",
    "\n",
    "!helm template demo-helm-201-common $wd_init/demo-helm-201 -f $wd_init/demo-helm-201/values.yaml --output-dir=work/out/common"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "* Templating, Variables\n",
    "  * `.Values.*` holds all variables (from file and command line)\n",
    "  * `values.yaml` and any additional values-file will be merged\n",
    "\n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Action (01)\n",
    "  * `image.version`\n",
    "  * env-specific values file\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Render template and generate Kubernetes resource files for *TEST* stage\"\n",
    "\n",
    "!helm template demo-helm-201-test $wd_init/demo-helm-201 -f $wd_init/demo-helm-201/values.test.yaml --output-dir=work/out/test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "* Templating, Functions\n",
    "  * `_helpers.tpl` holds set of helper functions used in template files\n",
    "  * common Go template functions are included (and, or, len, ...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "* Templating, Functions\n",
    "  * `with` set the scope\n",
    "  * `range` iterate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Action (02)\n",
    "  * new `sa.yaml`\n",
    "  * values in `values.test.yaml`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Render template and generate Kubernetes resource files for *TEST* stage\"\n",
    "\n",
    "!helm template demo-helm-201-test $wd_init/demo-helm-201 -f $wd_init/demo-helm-201/values.test.yaml --output-dir=work/out/test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "* Charts and Subcharts\n",
    "  * Subchart is a standalone chart\n",
    "  * only parent knows the subcharts / childrens\n",
    "  * ...and only parent can override fields"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Action \n",
    "  * checkout existing chart\n",
    "  * create a new chart in `parent/chart` directory\n",
    "  * configure dependency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Print out existing chart (without subchart)\"\n",
    "\n",
    "!tree $wd_init"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Create new (sub)chart in the *charts* dir\"\n",
    "\n",
    "!helm create $wd_init/demo-helm-201/charts/demo-subchart\n",
    "!rm -rf $wd_init/demo-helm-201/charts/demo-subchart/templates/*\n",
    "\n",
    "!tree $wd_init"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Action (03)\n",
    "  * template for `ConfigMap` in subchart\n",
    "  * value file in subchart\n",
    "  * override options"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Dry-Run subchart - with local value file\"\n",
    "\n",
    "!helm install --generate-name --dry-run --debug $wd_init/demo-helm-201/charts/demo-subchart"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Render entire chart - with *pre* value file, to override subchart fields\"\n",
    "\n",
    "!helm template demo-helm-201-pre $wd_init/demo-helm-201 -f $wd_init/demo-helm-201/values.pre.yaml --output-dir=work/out/pre"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Render entire chart - with *test* value file, to *NOT* override subchart fields\"\n",
    "\n",
    "!helm template demo-helm-201-test $wd_init/demo-helm-201 -f $wd_init/demo-helm-201/values.test.yaml --output-dir=work/out/test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "\n",
    "## Life Cycle and Hooks\n",
    "\n",
    "Hooks allows to execute specific resource definitions at defined points in the Helm life cycle.\n",
    "Available hooks\n",
    "\n",
    "* `pre-install`\n",
    "* `post-install`\n",
    "* `pre-delete`\n",
    "* `post-delete`\n",
    "* `pre-upgrade`\n",
    "* `post-upgrade`\n",
    "* `pre-rollback`\n",
    "* `post-rollback`\n",
    "* `test`\n",
    "\n",
    "Examples\n",
    "\n",
    "* prepare the installation and create specific resources beforehand (ConfigMap, Job etc)\n",
    "* clean up database before uninstalling application\n",
    "* return a license or deregister/unsubscribe\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-Install and Post-Delete\n",
    "\n",
    "Example for `pre-install` and `post-delete` hook. Hooks are usual Kubernetes resource definitions with special annotations.\n",
    "\n",
    "```\n",
    "metadata:\n",
    "  annotations:\n",
    "    \"helm.sh/hook\": pre-install, post-delete\n",
    "    \"helm.sh/hook-weight\": \"-5\"\n",
    "    \"helm.sh/hook-delete-policy\": hook-succeeded\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Create new (sub)chart in the *charts* dir\"\n",
    "\n",
    "!helm create $wd_init/demo-helm-201/charts/demo-hook\n",
    "!rm -rf $wd_init/demo-helm-201/charts/demo-hook/templates/ingress.yaml\n",
    "!rm -rf $wd_init/demo-helm-201/charts/demo-hook/templates/service.yaml\n",
    "!rm -rf $wd_init/demo-helm-201/charts/demo-hook/templates/tests\n",
    "\n",
    "\n",
    "!tree $wd_init"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "* Action (04)\n",
    "  * create 2 jobs for different hooks\n",
    "  * install and upgrade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Install subchart for hook testing\"\n",
    "\n",
    "!echo \"some OCP permissions adjustments (for nginx)...\"\n",
    "!oc adm policy add-scc-to-user anyuid -z demo-hook-dev\n",
    "\n",
    "!helm upgrade --install demo-hook-dev $wd_init/demo-helm-201/charts/demo-hook -n demo-helm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Update subchart for hook testing now with job definitions\"\n",
    "\n",
    "!helm upgrade --install demo-hook-dev $wd_init/demo-helm-201/charts/demo-hook -n demo-helm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "* Action\n",
    "  * current `pods` and `jobs`\n",
    "  * events\n",
    "  * clean-up and check out the job with `post-delete`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Delete subchart which triggers hook\"\n",
    "\n",
    "!helm delete demo-hook-dev -n demo-helm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "\n",
    "## OCI Registry\n",
    "\n",
    "Helm 3 provides the option to store helm charts not only on a HTTP server, but also in a OCI Registry. This allows the option to hold the artifacts (container image and helm charts) in the same registry."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testing\n",
    "\n",
    "* Action\n",
    "  * use a local registry\n",
    "  * push a local helm chart\n",
    "  * install from OCI registry\n",
    "\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Run a local registry\"\n",
    "\n",
    "!docker run -dp 5000:5000 --restart=always --name registry registry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Create helm package from the hook subchart\"\n",
    "\n",
    "!echo \"...copy subchart in own dir for testing...\"\n",
    "!cp -r $wd_init/demo-helm-201/charts/demo-hook $wd_init/\n",
    "\n",
    "!echo \"...helm package...\"\n",
    "!helm package $wd_init/demo-hook --version 1.0.3 --destination $wd_init\n",
    "!helm package $wd_init/demo-hook --version 1.0.4 --destination $wd_init"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Push helm package to registry\"\n",
    "\n",
    "!echo \"...helm registry login...\"\n",
    "!helm registry login -u testuser -p testpassword localhost:5000\n",
    "\n",
    "!echo \"...helm package push...\"\n",
    "!helm push $wd_init/demo-hook-1.0.3.tgz oci://localhost:5000/helm-charts\n",
    "!helm push $wd_init/demo-hook-1.0.4.tgz oci://localhost:5000/helm-charts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Verify helm package in OCI registry\"\n",
    "\n",
    "!echo \"...show chart details...\"\n",
    "!helm show chart oci://localhost:5000/helm-charts/demo-hook\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Install (render) helm package directly from OCI registry\"\n",
    "\n",
    "!echo \"...render existing chart from registry...\"\n",
    "!helm template release-demo-hook-oci oci://localhost:5000/helm-charts/demo-hook --version 1.0.3 --output-dir=work/out/oci\n",
    "\n",
    "!echo \"...try to render non-existing chart from registry...\"\n",
    "!helm template release-demo-hook-oci oci://localhost:5000/helm-charts/demo-hook --version 2.2.2 --output-dir=work/out/oci\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "---\n",
    "\n",
    "## Misc\n",
    "\n",
    "Some additional details\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Internal/Release State\n",
    "\n",
    "With Helm3 not Tiller exists anymore. All configuration and release state, including value files, is now stored in a `Secret` instead of a `ConfigMap`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Install subchart\"\n",
    "\n",
    "!helm upgrade --install release-demo-hook-state $wd_init/demo-helm-201/charts/demo-hook -n demo-helm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"List all existing helm releases\"\n",
    "\n",
    "!helm list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Print out release information from Helm Release Secret\"\n",
    "\n",
    "!oc get secret sh.helm.release.v1.release-demo-hook-state.v1 -o jsonpath='{.data.release}' | base64 --decode | base64 --decode | gunzip -c | jq"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helm Upgrade and force re-deployment\n",
    "\n",
    "Helm determines the relevant changes and updates only the necessary resources. In case the content of a `ConfigMap` changed, but not the image version, a redeployment or restart of the deployment will be not triggered.\n",
    "To trigger a redeployment a relation between `ConfigMap` content and `Deployment` is needed\n",
    "\n",
    "```\n",
    "kind: Deployment\n",
    "spec:\n",
    "  template:\n",
    "    metadata:\n",
    "      annotations:\n",
    "        checksum/config: {{ include (print $.Template.BasePath \"/configmap.yaml\") . | sha256sum }}\n",
    "```\n",
    "\n",
    "The idea is, that the `Deployment` contains an annotation with a checksum of the related `ConfigMap`. Any changes in the content, results in an other `sha256sum`, results in an other annotation, which results in modified `Deployment` definition."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IBM Cloud Container Registry \n",
    "\n",
    "The IBM Cloud Container Registry is OCI compliant and could also be used for Helm packages\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo \"Push chart to IBM CR\"\n",
    "\n",
    "!ibmcloud login --sso\n",
    "\n",
    "!ibmcloud cr login \n",
    "\n",
    "!echo \"...use API key to login...\"\n",
    "!helm registry login -u iamapikey\n",
    "\n",
    "!helm push $wd_init/demo-hook-1.0.1.tgz oci://de.icr.io/demo-helm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### VS Code Extensions\n",
    "\n",
    "* `yaml`\n",
    "* `kubernetes` understands also the helm template syntax"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "This was a walkthrough for Helm 201 with covering topics like\n",
    "\n",
    "* templating\n",
    "* charts and subcharts\n",
    "* OCI Registry"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "* [Helm - Storage Provider - Secret instead ConfigMap](https://helm.sh/docs/topics/advanced/#storage-backends)\n",
    "* [Helm - (OCI) Registry](https://helm.sh/docs/topics/registries/)\n",
    "* [Helm - Best Practices](https://helm.sh/docs/chart_best_practices/conventions/)\n",
    "* [Helm - Template Guide](https://helm.sh/docs/chart_template_guide/getting_started/)"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
  },
  "kernelspec": {
   "display_name": "Python 3.9.9 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}