Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod'

[hlherrera]

Version 2.8.1
Error
aws-mwaa-local-runner-2_8-local-runner-1 | [2024-04-22T00:05:58.985+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'

requirements.txt file:

--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.8.1/constraints-3.11.txt"

apache-airflow[cncf.kubernetes]==2.8.1
apache-airflow-providers-amazon==8.16.0
kubernetes==23.6.0
kubernetes-asyncio==24.2.3
cachetools==5.3.2

Command:
./mwaa-local-env start

Logs:

aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: annotated-types>=0.4.0 in ./.local/lib/python3.11/site-packages (from pydantic>=2.3.0->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (0.6.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: pydantic-core==2.14.6 in ./.local/lib/python3.11/site-packages (from pydantic>=2.3.0->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (2.14.6)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: docutils in ./.local/lib/python3.11/site-packages (from python-daemon>=3.0.0->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (0.20.1)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: text-unidecode>=1.3 in ./.local/lib/python3.11/site-packages (from python-slugify>=5.0->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.3)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: scramp<1.5.0,>=1.2.0 in ./.local/lib/python3.11/site-packages (from redshift_connector>=2.0.888->apache-airflow-providers-amazon==8.16.0->-r /usr/local/airflow/requirements/requirements.txt (line 4)) (1.4.4)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: beautifulsoup4<5.0.0,>=4.7.0 in ./.local/lib/python3.11/site-packages (from redshift_connector>=2.0.888->apache-airflow-providers-amazon==8.16.0->-r /usr/local/airflow/requirements/requirements.txt (line 4)) (4.12.2)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: lxml>=4.6.5 in ./.local/lib/python3.11/site-packages (from redshift_connector>=2.0.888->apache-airflow-providers-amazon==8.16.0->-r /usr/local/airflow/requirements/requirements.txt (line 4)) (5.1.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: charset-normalizer<4,>=2 in ./.local/lib/python3.11/site-packages (from requests->kubernetes==23.6.0->-r /usr/local/airflow/requirements/requirements.txt (line 5)) (3.3.2)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: idna<4,>=2.5 in ./.local/lib/python3.11/site-packages (from requests->kubernetes==23.6.0->-r /usr/local/airflow/requirements/requirements.txt (line 5)) (3.6)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: greenlet!=0.4.17 in ./.local/lib/python3.11/site-packages (from sqlalchemy<2.0,>=1.4.28->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (3.0.3)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: requests_toolbelt in ./.local/lib/python3.11/site-packages (from apache-airflow-providers-http->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.0.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in ./.local/lib/python3.11/site-packages (from httpx->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (0.16.3)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in ./.local/lib/python3.11/site-packages (from httpx->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.5.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: sniffio in ./.local/lib/python3.11/site-packages (from httpx->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.3.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: opentelemetry-exporter-otlp-proto-grpc==1.22.0 in ./.local/lib/python3.11/site-packages (from opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.22.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: opentelemetry-exporter-otlp-proto-http==1.22.0 in ./.local/lib/python3.11/site-packages (from opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.22.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: backoff<3.0.0,>=1.10.0 in ./.local/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.22.0->opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (2.2.1)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: googleapis-common-protos~=1.52 in ./.local/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.22.0->opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.62.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: grpcio<2.0.0,>=1.0.0 in ./.local/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.22.0->opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.60.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.22.0 in ./.local/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.22.0->opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.22.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: opentelemetry-proto==1.22.0 in ./.local/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.22.0->opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.22.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: opentelemetry-sdk~=1.22.0 in ./.local/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.22.0->opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (1.22.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: protobuf<5.0,>=3.19 in ./.local/lib/python3.11/site-packages (from opentelemetry-proto==1.22.0->opentelemetry-exporter-otlp-proto-grpc==1.22.0->opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (4.25.2)
aws-mwaa-local-runner-2_8-local-runner-1  | Collecting oauthlib>=3.0.0
aws-mwaa-local-runner-2_8-local-runner-1  |   Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
aws-mwaa-local-runner-2_8-local-runner-1  |      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 26.9 MB/s eta 0:00:00
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: soupsieve>1.2 in ./.local/lib/python3.11/site-packages (from beautifulsoup4<5.0.0,>=4.7.0->redshift_connector>=2.0.888->apache-airflow-providers-amazon==8.16.0->-r /usr/local/airflow/requirements/requirements.txt (line 4)) (2.5)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: pycparser in ./.local/lib/python3.11/site-packages (from cffi>=1.12->cryptography>=0.9.3->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (2.21)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: dnspython>=1.15.0 in ./.local/lib/python3.11/site-packages (from email-validator<2,>=1.0.5->flask-appbuilder==4.3.10->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (2.4.2)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: Babel>=2.3 in ./.local/lib/python3.11/site-packages (from Flask-Babel<3,>=1->flask-appbuilder==4.3.10->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (2.14.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: limits>=2.8 in ./.local/lib/python3.11/site-packages (from Flask-Limiter<4,>3->flask-appbuilder==4.3.10->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (3.7.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: ordered-set<5,>4 in ./.local/lib/python3.11/site-packages (from Flask-Limiter<4,>3->flask-appbuilder==4.3.10->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (4.1.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: h11<0.15,>=0.13 in ./.local/lib/python3.11/site-packages (from httpcore<0.17.0,>=0.15.0->httpx->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (0.14.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: anyio<5.0,>=3.0 in ./.local/lib/python3.11/site-packages (from httpcore<0.17.0,>=0.15.0->httpx->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (4.2.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: zipp>=0.5 in ./.local/lib/python3.11/site-packages (from importlib-metadata<7.0,>=6.0->opentelemetry-api>=1.15.0->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (3.17.0)
aws-mwaa-local-runner-2_8-local-runner-1  | Collecting pyasn1<0.6.0,>=0.4.6
aws-mwaa-local-runner-2_8-local-runner-1  |   Downloading pyasn1-0.5.1-py2.py3-none-any.whl (84 kB)
aws-mwaa-local-runner-2_8-local-runner-1  |      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.9/84.9 kB 10.1 MB/s eta 0:00:00
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: asn1crypto>=1.5.1 in ./.local/lib/python3.11/site-packages (from scramp<1.5.0,>=1.2.0->redshift_connector>=2.0.888->apache-airflow-providers-amazon==8.16.0->-r /usr/local/airflow/requirements/requirements.txt (line 4)) (1.5.1)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: importlib-resources>=1.3 in ./.local/lib/python3.11/site-packages (from limits>=2.8->Flask-Limiter<4,>3->flask-appbuilder==4.3.10->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (6.1.1)
aws-mwaa-local-runner-2_8-local-runner-1  | Requirement already satisfied: opentelemetry-semantic-conventions==0.43b0 in ./.local/lib/python3.11/site-packages (from opentelemetry-sdk~=1.22.0->opentelemetry-exporter-otlp-proto-grpc==1.22.0->opentelemetry-exporter-otlp->apache-airflow[cncf.kubernetes]==2.8.1->-r /usr/local/airflow/requirements/requirements.txt (line 3)) (0.43b0)
aws-mwaa-local-runner-2_8-local-runner-1  | Installing collected packages: websocket-client, pyasn1, oauthlib, cachetools, rsa, requests-oauthlib, pyasn1-modules, kubernetes-asyncio, google-auth, kubernetes
aws-mwaa-local-runner-2_8-local-runner-1  | Successfully installed cachetools-5.3.2 google-auth-2.26.2 kubernetes-23.6.0 kubernetes-asyncio-24.2.3 oauthlib-3.2.2 pyasn1-0.5.1 pyasn1-modules-0.3.0 requests-oauthlib-1.3.1 rsa-4.9 websocket-client-1.7.0
aws-mwaa-local-runner-2_8-local-runner-1  | 
aws-mwaa-local-runner-2_8-local-runner-1  | [notice] A new release of pip available: 22.3.1 -> 24.0
aws-mwaa-local-runner-2_8-local-runner-1  | [notice] To update, run: pip install --upgrade pip

Inside the runtime container:

aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:02:23.092+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:02:52.721+0000] {{triggerer_job_runner.py:481}} INFO - 0 triggers currently running
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:02:53.658+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:03:24.282+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:03:52.931+0000] {{triggerer_job_runner.py:481}} INFO - 0 triggers currently running
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:03:55.105+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:04:26.153+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:04:53.206+0000] {{triggerer_job_runner.py:481}} INFO - 0 triggers currently running
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:04:57.234+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:05:27.988+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:05:53.433+0000] {{triggerer_job_runner.py:481}} INFO - 0 triggers currently running
aws-mwaa-local-runner-2_8-local-runner-1  | [2024-04-22T00:05:58.985+0000] {{processor.py:376}} WARNING - Error when trying to pre-import module 'airflow.providers.cncf.kubernetes.operators.pod' found in /usr/local/airflow/dags/jupyter_notebook_poc.py: No module named 'airflow.providers.cncf'

Trying to load this dag:

from airflow.providers.cncf.kubernetes.operators.pod import KubernetesPodOperator

k = KubernetesPodOperator(
    name="hello-dry-run",
    image="debian",
    cmds=["bash", "-cx"],
    arguments=["echo", "10"],
    labels={"foo": "bar"},
    task_id="dry_run_demo",
    do_xcom_push=True,
)

k.dry_run()