Airflow 3.0.6 - Kerberos issue when running beeline

[msestak]

Hi,

We are running this task in airflow 2.4.3 and it suceeds. beeline finds the ticket generated and it connects to Impala.

 t1 = BashOperator(task_id="impala_start_setDateTable", bash_command="bash -c '/global/airflow/krb_2.sh'")

#!/bin/bash
set -euo pipefail

echo "=== IMPALA WRAPPER START ==="
hostname

CACHE="/tmp/krb5cc_${USER}_$(uuidgen)"
export KRB5CCNAME="FILE:${CACHE}"
echo "[INFO] Using KRB5CCNAME=$KRB5CCNAME"
rm -f "${CACHE}"

echo "[INFO] Running kinit..."
kinit -c "$KRB5CCNAME" -kt /global/airflow/bd.bdscloudera.keytab bd@SIT.PBZ.HR

echo "[INFO] Kerberos ticket:"
klist -c "$KRB5CCNAME" || { echo "[ERROR] No Kerberos ticket!"; exit 1; }

export HADOOP_OPTS="-Dsun.security.krb5.debug=true \
  -Djavax.security.auth.useSubjectCredsOnly=false \
  -Dsun.security.krb5.disableReferrals=true"
echo "[INFO] HADOOP_OPTS: $HADOOP_OPTS"

IMPALA_JDBC="jdbc:hive2://bdsclbaseimp.sit.pbz.hr:21050/default;principal=impala/bdsclbaseimp.sit.pbz.hr@SIT.PBZ.HR;ssl=true"
SQL_FILE="/global/airflow/dags_bdscloudera/batch_obrade/fmdl/scripts/insert_into_date_table.sql"

echo "[INFO] Running beeline..."
beeline -u "$IMPALA_JDBC" -f "$SQL_FILE"

echo "=== IMPALA WRAPPER DONE ==="

In Airflow 3.0.6 beeline command ignores generated Kerberos ticket and fails with error that Kerberos ticket is not found:

[2025-11-24, 13:13:19] INFO - Filling up the DagBag from /global/airflow/dags_bdscloudera/batch_obrade/fmdl/fmdl.py: source="airflow.models.dagbag.DagBag"
[2025-11-24, 13:13:20] INFO - Tmp dir root location: /tmp: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - Running command: ['/usr/bin/bash', '-c', "bash -c '/global/airflow/krb_2.sh'"]: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - Output:: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - === IMPALA WRAPPER START ===: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - bdsairflowcl03.sit.pbz.hr: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - [INFO] Using KRB5CCNAME=FILE:/tmp/krb5cc_airflow_c026c146-1499-4606-8437-7532b8a39798: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - [INFO] Running kinit...: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - [INFO] Kerberos ticket:: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - Ticket cache: FILE:/tmp/krb5cc_airflow_c026c146-1499-4606-8437-7532b8a39798: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - Default principal: bd@SIT.PBZ.HR: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - : source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - Valid starting       Expires              Service principal: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - 11/24/2025 14:13:20  11/25/2025 00:13:20  krbtgt/SIT.PBZ.HR@SIT.PBZ.HR: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - 	renew until 12/01/2025 14:13:20: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - [INFO] HADOOP_OPTS: -Dsun.security.krb5.debug=true   -Djavax.security.auth.useSubjectCredsOnly=false   -Dsun.security.krb5.disableReferrals=true: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:20] INFO - [INFO] Running beeline...: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:21] INFO - Connecting to jdbc:hive2://bdsclbaseimp.sit.pbz.hr:21050/default;principal=impala/bdsclbaseimp.sit.pbz.hr@SIT.PBZ.HR;ssl=true: source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"
[2025-11-24, 13:13:21] INFO - 14:13:21.596 [main] ERROR org.apache.thrift.transport.TSaslTransport - SASL negotiation failure:
...
[2025-11-24, 13:13:21] INFO - Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt): source="airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook"

We need help why this fails in Airflow 3 and how to use it in new Airflow.

[msestak]

We manged to make it work by passing env to BashOperator:
empty_env = {"EMPTY": ""}
t1 = BashOperator(task_id="impala_start_setDateTable", bash_command="bash -c '/global/airflow/krb_2.sh'", env=empty_env)

Is this a bug with airflow passing environment to tasks?