Not able to see Kubernetes jobs logs

[arpanm96]

Context

I have setup a deployment using the helm chart provided here, with the example DAGs loaded.

Everything is running fine and well. However, I am not able to see the logs produced by the DAG jobs. I can only see the worker pod logs.

I am using the sample kubernetes executor file with minimal changes.

Question

How to enable the job logs view in AirflowUI?

Attaching pictures for reference

• Using the K8s Executor, not able to see job logs

• Using the CeleryExecutor, able to see the job logs

Notes

While testing this,

• I was able to go inside the worker pod and verified that the log file for the job exists inside the pod.
• I exposed the port 8793 in the worker pod, but looks like the Airflow UI is not trying to fetch the worker logs from the logs location and neither the worker port to expose logs, unlike how it works for CeleryExecutor

Example screenshot for Airflow webserver pulling logs from an worker run using CeleryExecutor

Source for the above image - https://www.cloudwalker.io/wp-content/uploads/2019/09/image-36.png