Scheduler Logging Unimplemented in Helm Chart with Airflow V2 & SequentialExecutor (serve_log)Β #14222
Description
Problem
The helm chart does not implement a way for SequentialExecutor Airflow 2 deployments to serve logs; without using elasticsearch.
Details
Prior implementations utilize the CLI function serve_logs. This function has been deprecated as of v2.
In airflow/templates/scheduler/scheduler-deployment.yaml:
181 {{- if and $local (not $elasticsearch) }}
182 # Start the sidecar log server if we're in local mode and
183 # we don't have elasticsearch enabled.
184 - name: scheduler-logs
185 image: {{ template "airflow_image" . }}
186 imagePullPolicy: {{ .Values.images.airflow.pullPolicy }}
187 args: ["serve_logs"]This will cause the helm deployment to break; and the scheduler will perpetually fail to start the scheduler-logs container inside of the scheduler deployment.
Snippet from airflow upgrade guide.
Remove serve_logs command from CLI
The serve_logs command has been deleted. This command should be run only by internal application mechanisms and there is no need for it to be accessible from the CLI interface.
Partial Solution
Not sure how the non-elastic method for serving logs going forward.
Astronomer branches yaml by:
{{- if semverCompare ">=1.10.12" .Values.airflowVersion }}
...
{{- else }}
...
{{- end }}