Airflow DAGs fail with ModuleNotFoundError for internal package when using Astro Runtime Docker Image

[dattatejaofficial]

I am running Apache Airflow using Astronomer Astro Runtime in Docker. The Airflow UI starts correctly, but DAG parsing fails with:

ModuleNotFoundError: No module named 'currentsystem'
Here currentsystem is the system I am building currently. This is an internal Python Package from the same repository.

Project Structure
repo-root/
│
├── setup.py
├── currentsystem/ # internal python package
│ └── init.py
│
├── monitoring/ # airflow project directory
│ └── dags/
│ └── monitoring_pipeline.py
│
└── Dockerfile

The existing Dockerfile consists of FROM quay.io/astronomer/astro-runtime:13.4.0 & later updated to

FROM quay.io/astronomer/astro-runtime:13.4.0

WORKDIR /usr/local/airflow

COPY setup.py ./
COPY currentsystem/ ./currentsystem/
COPY monitoring/ ./monitoring/

RUN pip install -e .

ENV AIRFLOW__CORE__DAGS_FOLDER=/usr/local/airflow/monitoring/dags

Is there a recommended way in Astro Runtime to:

• include an internal Python package from the same repo.
• make it importable inside DAGs.

[potiuk]

It's a problem with Astro vmost likely - you need to make sure that your dags folder is added to the PYTHONPATH - I think others raised similar issues - you have to check with astro

[dattatejaofficial]

Could you please explain the solution in detail or provide any references?

[potiuk]

No, I can't - it was somewhere on slack - you might look yourself in the Astronomer channel.

But you should definitely check with astro project, not here. Astro is managed by Astronomer and has it's own support and they know better about