Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support renamed Kubernetes configuration in Airflow 2.5.0 #677

Closed
3 of 4 tasks
emartgu opened this issue Dec 7, 2022 · 1 comment · Fixed by #719
Closed
3 of 4 tasks

Support renamed Kubernetes configuration in Airflow 2.5.0 #677

emartgu opened this issue Dec 7, 2022 · 1 comment · Fixed by #719
Labels
kind/enhancement kind - new features or changes
Milestone

Comments

@emartgu
Copy link

emartgu commented Dec 7, 2022

Checks

Motivation

Airflow config section kubernetes renamed to kubernetes_executor in 2.5.0:
https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html#airflow-config-section-kubernetes-renamed-to-kubernetes-executor-26873

This is causing warnings for now and might not work for later versions.

Implementation

Introducing an if statement to pick the correct env variable names based on Airflow version in this section:

## Airflow Configs (Kubernetes)

Example (with introduced airflow.version value):

  {{- if semverCompare "<2.5" .Values.airflow.version }}
    {{- if not .Values.airflow.config.AIRFLOW__KUBERNETES__NAMESPACE }}
  AIRFLOW__KUBERNETES__NAMESPACE: {{ .Release.Namespace | toString | b64enc | quote }}
    {{- end }}
  {{- else -}}
    {{- if not .Values.airflow.config.AIRFLOW__KUBERNETES_EXECUTOR__NAMESPACE }}
  AIRFLOW__KUBERNETES_EXECUTOR__NAMESPACE: {{ .Release.Namespace | toString | b64enc | quote }}
    {{- end }}
  {{- end }}

Are you willing & able to help?

  • I am able to submit a PR!
  • I can help test the feature!
@emartgu emartgu added the kind/enhancement kind - new features or changes label Dec 7, 2022
@thesuperzapper thesuperzapper added this to the airflow-8.7.0 milestone Dec 21, 2022
@thesuperzapper thesuperzapper added this to Unsorted in Issue Triage and PR Tracking via automation Dec 21, 2022
@thesuperzapper thesuperzapper moved this from Unsorted to Triage | Needs Investigation in Issue Triage and PR Tracking Dec 21, 2022
@thesuperzapper
Copy link
Member

@emartgu I want to avoid the need to explicitly set an airflow.version value, we can probably do something similar to what we did for logging.worker_log_server_port replacing celery.worker_log_server_port in airflow 2.2.0, and just include both configs.

This should suppress the warning logs, and future-proof us in case they remove the old one.

Issue Triage and PR Tracking automation moved this from Triage | Needs Investigation to Done Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement kind - new features or changes
Development

Successfully merging a pull request may close this issue.

2 participants