Airflow Chart - Service token file does not exist

[thispejo]

Name and Version

bitnami/airflow/16.3.0

What architecture are you using?

None

What steps will reproduce the bug?

helm install airflow-homo oci://registry-1.docker.io/bitnamicharts/airflow -f kubernetes\airflow\airflow_homo_bitnami.yaml --namespace=airflow-homo  --version 16.3.0

requiriments.txt
authlib
discord-webhook
kubernetes>=21.7.0,<24
apache-airflow-providers-apache-spark
pytz

Are you using any custom parameters or values?

auth:
  username: "xpto" 

executor: "CeleryKubernetesExecutor"

scheduler:
  replicaCount: 1
  autoscaling:
    enable: false     
  nodeSelector:
    name: Airflow 
  extraVolumes:
    - name: requirements
      configMap:
        name: requirements
  extraVolumeMounts:
    - name: requirements
      mountPath: /bitnami/python/requirements.txt
      subPath: requirements.txt

serviceAccount:
  create: true
  name: "airflow-service-account" 
  automountServiceAccountToken: true

  resources:
    requests:
      cpu: "1500m" 
      memory: "4Gi" 
    limits:
      cpu: "1600m" 
      memory: "5Gi"
  readinessProbe:
    enabled: true
    initialDelaySeconds: 90
    timeoutSeconds: 15
    periodSeconds: 20
    failureThreshold: 15

What do you see instead?

[2024-02-24T12:54:07.802+0000] {db.py:1750} INFO - Connection successful.
airflow-scheduler 12:54:08.17 INFO  ==> Waiting for Airflow Webserver to be up
airflow-scheduler 12:54:08.18 INFO  ==> ** Airflow setup finished! **

airflow-scheduler 12:54:08.21 INFO  ==> ** Starting Airflow **
/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/cli/cli_config.py:926 DeprecationWarning: The namespace option in [kubernetes] has been moved to the namespace option in [kubernetes_executor] - the old setting has been used, but please update your config.
/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:107 DeprecationWarning: The namespace option in [kubernetes] has been moved to the namespace option in [kubernetes_executor] - the old setting has been used, but please update your config.
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
[2024-02-24T12:54:10.379+0000] {task_context_logger.py:63} INFO - Task context logging is enabled
/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/kube_config.py:37 DeprecationWarning: The pod_template_file option in [kubernetes] has been moved to the pod_template_file option in [kubernetes_executor] - the old setting has been used, but please update your config.
/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/kube_config.py:39 DeprecationWarning: The delete_worker_pods option in [kubernetes] has been moved to the delete_worker_pods option in [kubernetes_executor] - the old setting has been used, but please update your config.
/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/kube_config.py:46 DeprecationWarning: The worker_container_repository option in [kubernetes] has been moved to the worker_container_repository option in [kubernetes_executor] - the old setting has been used, but please update your config.
/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/kube_config.py:47 DeprecationWarning: The worker_container_tag option in [kubernetes] has been moved to the worker_container_tag option in [kubernetes_executor] - the old setting has been used, but please update your config.
/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/kube_config.py:57 DeprecationWarning: The namespace option in [kubernetes] has been moved to the namespace option in [kubernetes_executor] - the old setting has been used, but please update your config.
/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/kube_config.py:71 DeprecationWarning: The namespace option in [kubernetes] has been moved to the namespace option in [kubernetes_executor] - the old setting has been used, but please update your config.
[2024-02-24T12:54:10.562+0000] {scheduler_job_runner.py:808} INFO - Starting the scheduler
[2024-02-24T12:54:10.562+0000] {scheduler_job_runner.py:815} INFO - Processing each file at most -1 times
[2024-02-24T12:54:10.563+0000] {kubernetes_executor.py:300} INFO - Start Kubernetes executor
[2024-02-24T12:54:10.566+0000] {scheduler_job_runner.py:872} ERROR - Exception when executing SchedulerJob._run_scheduler_loop
Traceback (most recent call last):
  File "/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/jobs/scheduler_job_runner.py", line 846, in _execute
    self.job.executor.start()
  File "/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/celery/executors/celery_kubernetes_executor.py", line 112, in start
    self.kubernetes_executor.start()
  File "/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py", line 308, in start
    self.kube_client = get_kube_client()
                       ^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/airflow/venv/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/kube_client.py", line 128, in get_kube_client
    config.load_incluster_config(client_configuration=configuration)
  File "/opt/bitnami/airflow/venv/lib/python3.11/site-packages/kubernetes/config/incluster_config.py", line 121, in load_incluster_config
    try_refresh_token=try_refresh_token).load_and_set(client_configuration)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/airflow/venv/lib/python3.11/site-packages/kubernetes/config/incluster_config.py", line 54, in load_and_set
    self._load_config()
  File "/opt/bitnami/airflow/venv/lib/python3.11/site-packages/kubernetes/config/incluster_config.py", line 73, in _load_config
    raise ConfigException("Service token file does not exist.")
kubernetes.config.config_exception.ConfigException: Service token file does not exist.

Additional information

Starting from version 16.3.0, when using the "CeleryKubernetesExecutor" executor, there began to be issues with the pod scheduler. This problem does not occur in version 16.2.1.

[ebizboy]

add scheduler.automountServiceAccountToken : true

[thispejo]

I'm still encountering the same issue even after setting scheduler.automountServiceAccountToken: true. I also tried configuring the serviceAccount with create: false and name: "default", but I'm facing the same error. I'm using Oracle Kubernetes Engine (OKE) for this setup. Any insights or suggestions would be greatly appreciated.

[aledec]

Have face same issue from an upgrade from 16.1.7 to 16.8.3, but with 16.2.1 not having this issue.

[andresbono]

Please, try with worker.automountServiceAccountToken=true if your use case requires it. Does it work now?

This was probably caused by #22382, a PR addressing security improvements for the chart.

[github-actions]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[github-actions]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

[thispejo]

working!

Thank's

it was necessary to put it on the web and in the scheduler too