Template extra volumes in helm chart

[planoe]

This adds the possibility to define dynamic names (defined in custom helm template functions) for volumes and volumeMounts definitions such as secret names, config map names, mount paths etc.

I did not find a way to add tests in which I could define custom template functions, but I ran the chart tests successfully so it brings no regressions and I already use this in production, which works fine so far.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[planoe]

@jedcunningham @dstandish what do you think about this feature ? I think it would be a valuable addition to make the chart more flexible. Many customers have different naming patterns in their K8S environment.

[potiuk]

I think this one needs documentation explaining usage of this feature. Otherwise it won't be discoverable.

[planoe]

I think this one needs documentation explaining usage of this feature. Otherwise it won't be discoverable.

Fair point. I suggest to add a comment/example above the impacted fields in values.yaml (Not sure how many people actually read/use the schema.json file, I can image most of airflow admins check values.yaml first). For instance:

# Mount additional volumes into webserver. It can be templated like in the following example:
#   extraVolumes:
#     - name: my-templated-extra-volume
#       secret:
#          secretName: '{{ include "my_secret_template" . }}'
#          defaultMode: 0640
#          optional: true
#
#   extraVolumeMounts:
#     - name: my-templated-extra-volume
#       mountPath: "{{ .Values.my_custom_path }}"
#       readOnly: true

Would it be acceptable this way ? If so I will proceed with the changes

[potiuk]

I think we also need a small chapter in Production Guide: https://airflow.apache.org/docs/helm-chart/stable/production-guide.html with short explanation of the use case and why you would like to do it.

[jedcunningham]

My 2c, an example like @planoe suggested is enough. We can't put everything in the prod guide, it'd be way too long. We could consider documenting more advanced features, but we should do that as a separate effort I think.

[potiuk]

My 2c, an example like @planoe suggested is enough. We can't put everything in the prod guide, it'd be way too long. We could consider documenting more advanced features, but we should do that as a separate effort I think.

Fine for me.

[planoe]

Sorry for the delay, I updated the comments. Hopefully it can be merged now

[potiuk]

There ere merge conflicts now after reformatting of the chart templates . Need to be fixed

[planoe]

Thanks I fixed the merge conflicts.

[boring-cyborg]

Awesome work, congrats on your first merged pull request!

[jedcunningham]

Thanks @planoe! Congrats on your first commit 🎉

[planoe]

Thank you, hopefully more to come soon :)