Allow mounting the Airflow config as a directory to avoid the subPath mount race

[diegoalvarez-clarity]

Why

The chart mounts airflow.cfg (and airflow_local_settings.py) into every component and the KubernetesExecutor pod template via a subPath bind-mount of the config ConfigMap. subPath + ConfigMap hits a known Kubernetes race: on node/volume lifecycle events (kubelet volume reconstruction, node autoscaling) the pre-computed subPath source inode goes stale and runc fails the mount at container init:

error during container init: error mounting
".../volume-subpaths/config/base/1" to rootfs at "/opt/airflow/airflow.cfg"

The task fails with StartError and no application log (the container never starts). It's intermittent, cross-component, multi-node, and runtime-version-independent. See #33788.

What

Add an opt-in mountConfigAsDir (default false; existing behaviour unchanged). When true, the config ConfigMap is mounted as a directory at {airflowHome}/config (no subPath) and AIRFLOW_CONFIG is set to {airflowHome}/config/airflow.cfg. A directory-projected ConfigMap volume has no pre-computed subPath bind source, so it's immune to the race. airflow_local_settings.py keeps its default path, so local-settings loading is unaffected. The separate pod_template_file.yaml subPath mount is out of scope.

Tests

• test_mount_config_as_dir — flag on: directory mount, no cfg/local-settings subPath, AIRFLOW_CONFIG set, across all components.
• test_config_mounted_via_subpath_by_default — default behaviour preserved.

Notes

Backward compatible (default off).

This PR was created with the assistance of generative AI tools.

Refs: #33788

[salvamadrid]

+1

[Miretpl]

The task fails with StartError and no application log (the container never starts). It's intermittent, cross-component, multi-node, and runtime-version-independent. See #33788.

The solution to the issue mentioned here is basically not to mount the /opt/airflow directory as a whole, and the solution is mentioned in the discussion by Jarek. If you have the same case as mentioned in the discussion, you can mount a volume at /opt/airflow/config (or a different location) and set the desired env variables/config options accordingly.

[diegoalvarez-clarity]

Thanks @Miretpl — I think we're describing two different cases, so let me draw the line clearly.

The whole /opt/airflow scenario from #33788 is a user-supplied mount, and yes, mounting at /opt/airflow/config instead is the right fix there.

The race this PR targets is on the chart's own config mount. The shared airflow_config_mount helper unconditionally emits two subPath mounts — airflow.cfg and airflow_local_settings.py — onto the config volume. Those are what runc fails to bind under node/volume-lifecycle churn (kubelet volume reconstruction / node autoscaling):

error mounting ".../volume-subpaths/config/base/1" to rootfs at "/opt/airflow/airflow.cfg"

The container never starts, so the task hard-fails with no application log.

A values-level extraVolumeMounts at /opt/airflow/config is additive — it can't remove the helper's subPath mounts, so the race persists. There's no values-only way to drop them today, and that's the gap this PR closes.

mountConfigAsDir: true switches the shared helper from the two subPaths to a single directory mount at /opt/airflow/config + AIRFLOW_CONFIG. It's opt-in and default-off, so existing behavior is unchanged. helm template diff: subPath count 2 → 0, everything else byte-identical; airflow_local_settings.py stays at its default path via the directory mount.

[diegoalvarez-clarity]

Hi @jedcunningham @hussein-awala @jscheffl @bugraoz93 @Miretpl 👋

Gentle nudge on this one when you get a chance. To recap where it stands: the PR adds an opt-in mountConfigAsDir flag (default false, so existing behavior is unchanged) that mounts the config ConfigMap as a directory instead of via subPath binds, which avoids the stale-inode StartError race we hit during node reconstruction / autoscaling.

@Miretpl — I followed up on your question about #33788 above: this targets the chart's own airflow.cfg / airflow_local_settings.py subPath mounts rather than user-supplied ones, since values-only extraVolumeMounts can't remove the helper's subPath mounts. Happy to expand or adjust if that framing doesn't fully address your concern.

Tests cover both the new directory-mount path and the preserved subPath behavior. I'm glad to rebase or make changes to help this move forward — thanks for taking a look! 🙏