flux HelmRelease not launching Jobs equivalently to helm install

[otosky]

Newbie flux user here, apologies if I've missed something simple!

I'm getting different behavior between running helm install via the CLI and creating (what I believe is) an equivalent HelmRelease via flux on the official Apache Airflow Helm chart exposed here.

The issue I'm seeing with the flux-owned HelmRelease is that Kubernetes Jobs are not being launched as helm hooks after the Postgresql dependency is up-and-running, so the remainder of the Airflow Pods are stuck waiting to initialize.

This job in particular is what I'm expecting to launch: https://github.com/apache/airflow/blob/master/chart/templates/migrate-database-job.yaml.

Running the helm install ... command on the chart will install it without this issue.

release.yaml

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: airflow
  namespace: airflow
spec:
  interval: 5m
  chart:
    spec:
      chart: ./chart
      sourceRef:
        kind: GitRepository
        name: airflow
        namespace: flux-system
      interval: 1m
      valuesFile: ./chart/values.yaml

source.yaml

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: GitRepository
metadata:
  name: airflow
  namespace: flux-system
spec:
  interval: 1m
  url: https://github.com/apache/airflow.git
  ref:
    tag: 2.0.1
  ignore: |
    # exclude all
    /*
    # include charts directory
    !/chart/

Any ideas of what I may be doing wrong?

[otosky]

Figured out that the issue was my not fully understanding how Helm hooks work, and confusion around the Helm --wait flag.

In a nutshell, this chart has a migration Job that acts as a post-install/upgrade hook, but doesn't play well with the wait flag, since the Kubernetes resources that the chart declares have init-containers which wait on completion of the migrations job! Probably a chart design decision that they will have to re-evaluate, see apache/airflow#11979.

In any case, this was resolved by adding the disableWait parameter under HelmRelease spec.<action> like so:

release.yaml

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: airflow
  namespace: airflow
spec:
  interval: 5m
  chart:
    spec:
      chart: ./chart
      sourceRef:
        kind: GitRepository
        name: airflow
        namespace: flux-system
      interval: 1m
  install:
    disableWait: true
  upgrade:
    disableWait: true

[KishinNext]

Another way in the new helm version:

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
  name: airflow
  namespace: flux-system
spec:
  interval: 10m
  url: https://airflow.apache.org
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: airflow
  namespace: flux-system
spec:
  chart:
    spec:
      chart: airflow
      sourceRef:
        kind: HelmRepository
        name: airflow
        namespace: flux-system
  interval: 30s
  targetNamespace: dev
  install:
    disableWait: true
  upgrade:
    disableWait: true
  values:
    fernetKey: "RANDOMKEY"
    executor: "KubernetesExecutor"
    flower:
      enabled: False
    redis:
      enabled: False
    dags:
      persistence: 
        enabled: True
        size: 512Mi
        storageClassName: aws-efs
        existingClaim: efs-storage-claim
    extraEnv: |-
      - name: "AIRFLOW__KUBERNETES__RUN_AS_USER"
        value: "50000"
      - name: "AIRFLOW__CORE__LOAD_EXAMPLES"
        value: "True"
      - name: "AIRFLOW__WEBSERVER__SECRET_KEY"
        value: "RANDOMKEY"
      - name: "AIRFLOW__CORE__FERNET_KEY"
        value: "RANDOMKEY"
      - name: "AIRFLOW__KUBERNETES__DAGS_IN_IMAGE"
        value: "False"
      - name: "AIRFLOW__KUBERNETES__NAMESPACE"
        value: "dev"
      - name: "AIRFLOW__KUBERNETES__WORKER_CONTAINER_REPOSITORY"
        value: "apache/airflow"
      - name: "AIRFLOW__KUBERNETES__WORKER_CONTAINER_TAG"
        value: "2.2.3-python3.7"