-
Notifications
You must be signed in to change notification settings - Fork 16.6k
Description
Apache Airflow version
2.10.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
I configured Airflow with the following OpenTelemetry Traces configuration -
airflow:
image:
tag: 2.10.2-python3.12
extraPipPackages:
- "apache-airflow[otel]"
config:
AIRFLOW__TRACES__OTEL_DEBUGGING_ON: "True"
AIRFLOW__TRACES__OTEL_ON: "True"
AIRFLOW__TRACES__OTEL_HOST: "my_otel_host"
AIRFLOW__TRACES__OTEL_PORT: <my_otel_port>
AIRFLOW__TRACES__OTEL_SERVICE: "airflow"
The above configuration when applied should emit airflow traces to the configured otel host. But they are only getting emitted in the console and not to the configured otel host.
The reason for this is the otel_debugging_on attribute. When this is set to true, we see the above behaviour where the airflow does not emit traces to the host and only emits in the console. The airflow docs also do mention that it will emit traces to both console and the otel host.
If the attribute is removed from the otel traces config or is set to False we can see the traces emitted to the configured host.
Proposed Change -
The otel_debugging_on when set to true in otel traces configuration, the traces should get emitted to both the console and the configured otel host.
Airflow Version Used - 2.10.2
What you think should happen instead?
No response
How to reproduce
Configure your airflow with the following traces configuration and check your otel host for traces. The traces should be visible with the config applied -
AIRFLOW__TRACES__OTEL_ON: "True"
AIRFLOW__TRACES__OTEL_HOST: "my_otel_host"
AIRFLOW__TRACES__OTEL_PORT: <my_otel_port>
AIRFLOW__TRACES__OTEL_SERVICE: "airflow"
Now add the following attribute within the above configured traces config. Once this is added, the traces are no longer visible in the configured otel host but are visible in the console/logs.
AIRFLOW__TRACES__OTEL_DEBUGGING_ON: "True"
Operating System
Linux
Versions of Apache Airflow Providers
No response
Deployment
Other 3rd-party Helm chart
Deployment details
Used airflow community helm chart for deploying airflow.
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct