Airflow UI does not show triggerer logs during runtime under Kubernetes Executor

[manipatnam]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.10.1

What happened?

I created a simple DAG using TimeDeltaSensorAsync to test async sensor behavior across different Airflow executors and Astro Runtime versions. The DAG uses mode="reschedule" and a delta of 10 minutes.

When running this DAG:

With the Celery Executor (Astro Runtime 9.19.0):

The triggerer logs for TimeDeltaSensorAsync are visible in the Airflow UI under task logs during runtime.

These logs also appear inside the triggerer container.

With the Kubernetes Executor (Astro Runtime 9.19.0 and 12.1.0):

The triggerer logs are not visible in the Airflow UI during runtime.

However, the logs are present inside the triggerer container when inspected directly (e.g., via kubectl logs).

This suggests that the triggerer logs are not being forwarded or uploaded properly to the remote logging backend (in our case, S3) when using the Kubernetes Executor.

What you think should happen instead?

Runtime logs for tasks should be visible in the Airflow UI when using the Kubernetes Executor — just like they are with the Celery Executor. Logs from the initial task attempt should be uploaded to remote storage and available during runtime.

How to reproduce

Create a DAG using TimeDeltaSensorAsync with mode="reschedule" and a short delta, e.g., 10 minutes:

from airflow import DAG
from airflow.sensors.time_delta import TimeDeltaSensorAsync
from datetime import datetime, timedelta

with DAG(
    dag_id="test_timedelta_sensor_async",
    start_date=datetime(2025, 7, 15),
    schedule_interval=None,
    catchup=False,
) as dag:
    TimeDeltaSensorAsync(
        task_id="wait_for_time",
        delta=timedelta(minutes=10),
        mode="reschedule",
    )

Run this DAG using the Kubernetes Executor

Observe the task logs in the Airflow UI while the sensor is in deferred mode.

Operating System

debian

Versions of Apache Airflow Providers

No response

Deployment

Astronomer

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[wolfier]

I believe the trigger logs are generally only uploaded when the trigger is fired. Additionally, Airflow does reach out to the triggerer when there is an active trigger to fetch the logs. The issue is due to some misconfigured networking.