[chart/v1-2x-test] Helm Chart Documentation Update (#66970)

chart/docs/conf.py

@@ -38,6 +38,10 @@
 from typing import Any
 
 import yaml
+from airflow_breeze.global_constants import (
+    ALLOWED_KUBERNETES_VERSIONS,
+    HELM_VERSION,
+)
 from docs.utils.conf_constants import (
     AIRFLOW_FAVICON_PATH,
     AIRFLOW_REPO_ROOT_PATH,
@@ -300,6 +304,11 @@ def _process_array_items(items: dict, parent_name: str, param_name: str, default
         "package_name": PACKAGE_NAME,
         "package_version": PACKAGE_VERSION,
     },
+    "global_ctx": {
+        "package_version": PACKAGE_VERSION,
+        "min_k8s_version": ALLOWED_KUBERNETES_VERSIONS[0],
+        "helm_version": HELM_VERSION,
+    },
 }
 
 

chart/docs/customizing-labels.rst

@@ -21,7 +21,8 @@ Customizing Labels and Annotations for Pods
 Customizing Pod Labels
 ----------------------
 
-The Helm Chart allows you to customize labels for your Airflow objects. You can set global labels that apply to all objects and pods defined in the chart, as well as component-specific labels for individual Airflow components.
+The Helm Chart allows you to customize labels for your Airflow objects. You can set global labels that apply to all objects and pods defined in the chart,
+as well as component-specific labels for individual Airflow components.
 
 Global Labels
 ~~~~~~~~~~~~~

chart/docs/customizing-workers.rst

@@ -19,20 +19,18 @@ Customizing Workers
 ===================
 
 Both ``CeleryExecutor`` and ``KubernetesExecutor`` workers can be highly customized with the :ref:`workers parameters <parameters:workers>`.
-For example, to set resources on workers:
+For example, to set resources on workers for CeleryExecutor:
 
 .. code-block:: yaml
    :caption: values.yaml
 
    workers:
-     resources:
-       requests:
-         cpu: 1
-       limits:
-         cpu: 1
-
-One notable exception for ``KubernetesExecutor`` is that the default anti-affinity applied to ``CeleryExecutor`` workers to spread them across nodes
-is not applied to ``KubernetesExecutor`` workers, as there is no reason to spread out per-task workers.
+     celery:
+       resources:
+         requests:
+           cpu: 1
+         limits:
+           cpu: 1
 
 Custom ``pod_template_file``
 ----------------------------
@@ -43,8 +41,7 @@ worker types for ``CeleryKubernetesExecutor`` or if you need to customize someth
 
 .. note::
 
-   Some configuration options between Celery and Kubernetes workers can be overwritten by new ``workers.celery`` and ``workers.kubernetes`` sections.
-   Implementation of ``workers.celery`` and ``workers.kubernetes`` is not yet fully completed.
+   Configuration options between Celery and Kubernetes workers can be overwritten by ``workers.celery`` and ``workers.kubernetes`` sections.
 
 As an example, let's say you want to set ``priorityClassName`` on your workers:
 
@@ -71,3 +68,12 @@ As an example, let's say you want to set ``priorityClassName`` on your workers:
        priorityClassName: high-priority
        containers:
          - name: base
+
+The same can be achieved with default `pod_template_file`_ by overriding the ``priorityClassName`` option for KubernetesExecutor like:
+
+.. code-block:: yaml
+   :caption: values.yaml
+
+   workers:
+     kubernetes:
+       priorityClassName: high-priority

chart/docs/extending-the-chart.rst

@@ -21,7 +21,7 @@ Extending the Chart
 The Airflow Helm Chart can be easily extended by creating a custom chart which will depend on the Airflow chart.
 That can be useful in cases where there is a need for custom templates deployment (e.g. maintenance CronJobs),
 which are not directly related to the Airflow Helm Chart and should not be added to it in the source repository.
-During installation of custom chart, the Airflow chart will also be installed too.
+During installation of custom chart, the Airflow chart will also be installed.
 
 You can extend the official Airflow chart by applying the following steps.
 
@@ -55,17 +55,19 @@ This will give you the ability to add your custom templates without the need to
 In order to add the Airflow chart as a dependency (often called ``subcharts``) to your chart,
 add the following lines to your ``Chart.yaml`` file:
 
-.. code-block:: yaml
-   :caption: Chart.yaml
+.. jinja:: global_ctx
+
+   .. code-block:: yaml
+      :caption: Chart.yaml
 
-   dependencies:
-     - name: airflow
-       version: 1.11.0
-       repository: https://airflow.apache.org
+      dependencies:
+        - name: airflow
+          version: {{ package_version }}
+          repository: https://airflow.apache.org
 
 .. note::
 
-   Make sure you have already added the Airflow repo locally by running: ``helm repo add apache-airflow https://airflow.apache.org``.
+   Make sure that you have already added the Airflow repo locally by running: ``helm repo add apache-airflow https://airflow.apache.org``.
 
 .. tip::
 
@@ -76,14 +78,16 @@ Adding the Airflow chart as a dependency means that it will be deployed together
 You can disable the installation of Airflow by adding the ``condition`` field to the ``dependencies`` section
 like in the example below:
 
-.. code-block:: yaml
-   :caption: Chart.yaml
+.. jinja:: global_ctx
+
+   .. code-block:: yaml
+      :caption: Chart.yaml
 
-   dependencies:
-     - name: airflow
-       version: 1.11.0
-       repository: https://airflow.apache.org
-       condition: airflow.enabled
+      dependencies:
+        - name: airflow
+          version: {{ package_version }}
+          repository: https://airflow.apache.org
+          condition: airflow.enabled
 
 This will check if the value of ``airflow.enabled`` inside your ``values.yaml`` is ``true``.
 If it is, the Airflow chart will be deployed together with your custom chart.
@@ -101,7 +105,7 @@ you can download it by running the following command:
 
 .. note::
 
-   Make sure you are inside the directory which contains the ``Chart.yaml`` file.
+   Make sure that you are inside the directory which contains the ``Chart.yaml`` file.
 
 The chart will be downloaded and saved inside the ``charts/`` directory.
 

chart/docs/index.rst

@@ -62,26 +62,25 @@ deployment on a `Kubernetes <http://kubernetes.io>`__ cluster using the
 Requirements
 ------------
 
--  Kubernetes 1.30+ cluster
--  Helm 3.10+
--  PV provisioner support in the underlying infrastructure (optionally)
+.. jinja:: global_ctx
+
+   -  Kubernetes {{ min_k8s_version }}+ cluster
+   -  Helm {{ helm_version }}+
+   -  PV provisioner support in the underlying infrastructure (optionally)
 
 Features
 --------
 
 * Supported executors (all Airflow versions): ``LocalExecutor``, ``CeleryExecutor``, ``KubernetesExecutor``
 * Supported hybrid static executors (Airflow version ``2.11.X``): ``LocalKubernetesExecutor``, ``CeleryKubernetesExecutor``
 * Supported multiple Executors (``2.11+``)
-* Supported AWS executors with AWS provider version ``8.21.0+``:
+* Supported AWS executors with AWS provider version ``9.18.0+``:
 
    * ``airflow.providers.amazon.aws.executors.batch.AwsBatchExecutor``
    * ``airflow.providers.amazon.aws.executors.ecs.AwsEcsExecutor``
-
-* Supported AWS executors with AWS provider version ``9.9.0+``:
-
    * ``airflow.providers.amazon.aws.executors.aws_lambda.lambda_executor.AwsLambdaExecutor``
 
-* Supported Edge executor with edge3 provider version ``1.0.0+``:
+* Supported Edge executor with edge3 provider version ``1.6.0+``:
 
    * ``airflow.providers.edge3.executors.EdgeExecutor``
 
@@ -191,6 +190,10 @@ This will run database migrations every time there is a ``Sync`` event in Argo C
 
 If you use the ``CeleryExecutor`` or ``CeleryKubernetesExecutor`` with the built-in Redis, it is recommended that you set up a static Redis password either by supplying ``redis.passwordSecretName`` and ``data.brokerUrlSecretName`` or ``redis.password``.
 
+.. note::
+
+   Due to security concerns, it is not advised to set sensitive values like passwords within the ``values.yaml``.
+
 .. note::
 
    By default, Helm hooks are also enabled for ``extraSecrets`` or ``extraConfigMaps``. When using the above CI/CD tools, you might encounter issues due to these default hooks.
@@ -225,6 +228,10 @@ It is not enabled by default as this may cause unexpected behaviours on existing
 
 For existing installations, all your resources will be recreated with a new name and helm will delete previous resources.
 
+.. note::
+
+   In current ``useStandardNaming`` implementation, the standard name is not affecting Kubernetes Service Accounts created by Airflow Helm Chart.
+
 This won't delete existing PVCs for logs used by StatefulSets/Deployments, but it will recreate them with brand new PVCs.
 If you do want to preserve logs history you'll need to manually copy the data of these volumes into the new volumes after
 deployment. Depending on what storage backend/class you're using, this procedure may vary. If you don't mind starting

chart/docs/parameters-ref.rst

@@ -38,19 +38,21 @@ The following tables lists the configurable parameters of the Airflow chart and
         - Default
 
       {% for param in section["params"] %}
+         {% if "deprecated" not in param["description"] %}
+
       * - ``{{ param["name"] }}``
         - {{ param["description"] }}
         - ``{{ param["default"] }}``
-         {% if param["examples"] %}
+            {% if param["examples"] %}
 
           .. code-block:: yaml
              :caption: Examples
 
              {{ param["examples"] | indent(width=10) }}
 
+            {% endif %}
          {% endif %}
       {% endfor %}
-
    {% endfor %}
 
 
@@ -75,3 +77,37 @@ and install the chart:
 .. code-block:: bash
 
    helm install my-release apache-airflow/airflow --values override-values.yaml
+
+Deprecated Parameters
+=====================
+
+The following table contains all deprecated configuration parameters of the Airflow chart with their default values. All values defined in below table with be removed
+in the next Helm Chart major release.
+
+.. jinja:: params_ctx
+
+   .. list-table::
+      :widths: 15 10 30
+      :header-rows: 1
+
+      * - Parameter
+        - Description
+        - Default
+
+   {% for section in sections %}
+      {% for param in section["params"] %}
+         {% if "deprecated" in param["description"] %}
+      * - ``{{ param["name"] }}``
+        - {{ param["description"] }}
+        - ``{{ param["default"] }}``
+            {% if param["examples"] %}
+
+          .. code-block:: yaml
+             :caption: Examples
+
+             {{ param["examples"] | indent(width=10) }}
+
+            {% endif %}
+         {% endif %}
+      {% endfor %}
+   {% endfor %}

chart/docs/production-guide.rst

@@ -285,14 +285,19 @@ This setting can be configured in the Airflow chart at different levels:
    :caption: values.yaml
 
    workers:
-     safeToEvict: true
+     celery:
+       safeToEvict: true
+     kubernetes:
+       safeToEvict: true
    scheduler:
      safeToEvict: true
    apiServer:
      safeToEvict: true
 
-``workers.safeToEvict`` defaults to ``false``, and when using ``KubernetesExecutor``
-``workers.safeToEvict`` shouldn't be set to ``true`` as the workers may be removed before finishing.
+.. note::
+
+   ``workers.kubernetes.safeToEvict`` defaults to ``false`` as it shouldn't be set to ``true``,
+   because the tasks may be removed before finishing.
 
 Extending and customizing Airflow Image
 ---------------------------------------

chart/docs/quick-start.rst

@@ -26,11 +26,14 @@ Run Airflow on Kind cluster
 Install kind and create a cluster
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-We recommend testing with Kubernetes 1.30+, example:
 
-.. code-block:: bash
+.. jinja:: global_ctx
+
+   We recommend testing with Kubernetes {{ min_k8s_version }}+, example:
+
+   .. code-block:: bash
 
-   kind create cluster --image kindest/node:v1.30.13
+      kind create cluster --image kindest/node:{{ min_k8s_version }}
 
 Confirm it's up:
 

chart/docs/setting-resources-for-containers.rst

@@ -28,48 +28,26 @@ You can define different resources for various Airflow containers. By default th
 
 Possible containers where resources can be configured include:
 
-* Main Airflow containers and their sidecars. You can add the resources for these containers through the following parameters:
+.. jinja:: params_ctx
 
-   * ``workers.resources``
-   * ``workers.celery.logGroomerSidecar.resources``
-   * ``workers.kerberosSidecar.resources``
-   * ``workers.kerberosInitContainer.resources``
-   * ``scheduler.resources``
-   * ``scheduler.logGroomerSidecar.resources``
-   * ``dags.gitSync.resources``
-   * ``apiServer.resources``
-   * ``webserver.resources``
-   * ``flower.resources``
-   * ``dagProcessor.resources``
-   * ``dagProcessor.logGroomerSidecar.resources``
-   * ``triggerer.resources``
-   * ``triggerer.logGroomerSidecar.resources``
+   {% for section in sections %}
+      {% for param in section["params"] %}
+         {% if param["name"].endswith("resources") %}
+   * ``{{ param["name"] }}``
+         {% endif %}
+      {% endfor %}
+   {% endfor %}
 
-* Containers used for Airflow Kubernetes jobs or cron jobs. You can add the resources for these containers through the following parameters:
-
-   * ``cleanup.resources``
-   * ``createUserJob.resources``
-   * ``migrateDatabaseJob.resources``
-   * ``databaseCleanup.resources``
-
-* Other containers that can be deployed by the chart. You can add the resources for these containers through the following parameters:
-
-   * ``statsd.resources``
-   * ``pgbouncer.resources``
-   * ``pgbouncer.metricsExporterSidecar.resources``
-   * ``redis.resources``
-
-For example, specifying resources for worker Kerberos sidecar:
+For example, specifying resources for scheduler container:
 
 .. code-block:: yaml
    :caption: values.yaml
 
-   workers:
-     kerberosSidecar:
-       resources:
-         limits:
-           cpu: 200m
-           memory: 256Mi
-         requests:
-           cpu: 100m
-           memory: 128Mi
+   scheduler:
+      resources:
+      limits:
+         cpu: 1
+         memory: 1Gi
+      requests:
+         cpu: 500m
+         memory: 512Gi