Skip to content

otel_debugging_on emits traces ONLY to the console and not to the configured otel host #43427

@ninad-stackgen

Description

@ninad-stackgen

Apache Airflow version

2.10.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

I configured Airflow with the following OpenTelemetry Traces configuration -

airflow:
  image:
    tag: 2.10.2-python3.12
  extraPipPackages:
    - "apache-airflow[otel]"
  config:
    AIRFLOW__TRACES__OTEL_DEBUGGING_ON: "True"
    AIRFLOW__TRACES__OTEL_ON: "True"
    AIRFLOW__TRACES__OTEL_HOST: "my_otel_host"
    AIRFLOW__TRACES__OTEL_PORT: <my_otel_port>
    AIRFLOW__TRACES__OTEL_SERVICE: "airflow"

The above configuration when applied should emit airflow traces to the configured otel host. But they are only getting emitted in the console and not to the configured otel host.
The reason for this is the otel_debugging_on attribute. When this is set to true, we see the above behaviour where the airflow does not emit traces to the host and only emits in the console. The airflow docs also do mention that it will emit traces to both console and the otel host.
If the attribute is removed from the otel traces config or is set to False we can see the traces emitted to the configured host.

Proposed Change -
The otel_debugging_on when set to true in otel traces configuration, the traces should get emitted to both the console and the configured otel host.

Airflow Version Used - 2.10.2

What you think should happen instead?

No response

How to reproduce

Configure your airflow with the following traces configuration and check your otel host for traces. The traces should be visible with the config applied -

    AIRFLOW__TRACES__OTEL_ON: "True"
    AIRFLOW__TRACES__OTEL_HOST: "my_otel_host"
    AIRFLOW__TRACES__OTEL_PORT: <my_otel_port>
    AIRFLOW__TRACES__OTEL_SERVICE: "airflow"

Now add the following attribute within the above configured traces config. Once this is added, the traces are no longer visible in the configured otel host but are visible in the console/logs.

    AIRFLOW__TRACES__OTEL_DEBUGGING_ON: "True"

Operating System

Linux

Versions of Apache Airflow Providers

No response

Deployment

Other 3rd-party Helm chart

Deployment details

Used airflow community helm chart for deploying airflow.

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions