Skip to content

Standalone: set PYTHONPATH=AIRFLOW_HOME to match runtime image#2155

Merged
jeremybeard merged 1 commit into
mainfrom
fix-standalone-pythonpath
May 22, 2026
Merged

Standalone: set PYTHONPATH=AIRFLOW_HOME to match runtime image#2155
jeremybeard merged 1 commit into
mainfrom
fix-standalone-pythonpath

Conversation

@jeremybeard
Copy link
Copy Markdown
Contributor

Summary

Aligns standalone-mode PYTHONPATH with the production Astro runtime Docker image.

Standalone mode previously set PYTHONPATH=AIRFLOW_HOME/include only. That diverged from the runtime image, which sets PYTHONPATH=AIRFLOW_HOME (astro-runtime/image/Dockerfile#L19). This change brings standalone in line with the image.

Why

The previous behavior was introduced by #2060 with the stated intent of "mirroring the docker image" — but the docker image actually bakes AIRFLOW_HOME into PYTHONPATH, not AIRFLOW_HOME/include. The comment in the original code was also factually inaccurate. So in practice, imports that work in production (and pass astro deploy --parse) could fail when the same project was tested locally with astro dev start --standalone or astro dev pytest --standalone.

Concrete example that breaks under the current code but works in production:

# dags/hybrid_dag.py
from dags.my_blueprints import Extract  # works in prod, fails under --standalone

The prefixed from include.<module> import … pattern that the official docs recommend (Custom hooks and operators) continues to work both before and after this change.

Behavior change

  • from include.foo import bar — continues to work (include/ is findable as a namespace package under AIRFLOW_HOME).
  • from dags.foo import bar — now works (matches production).
  • ⚠️ import foo where foo.py lives flat in include/ — no longer resolves under standalone. This only ever worked under standalone (never in docker mode), so the breakage surface is small. The fix for users is to switch to from include.foo import …, which the docs already teach as the canonical pattern.

Test plan

  • go test ./airflow/ -run 'TestAirflow/TestStandaloneBuildEnv' (12 subtests, all pass)
  • Verified test is robust to inherited PYTHONPATH: PYTHONPATH=/some/inherited/path go test ./airflow/ -run 'TestAirflow/TestStandaloneBuildEnv$' passes (the test now calls t.Setenv("PYTHONPATH", "") so the strict equality assertion doesn't flake on developer/CI envs)
  • End-to-end: built a local binary, ran an integration test suite against it that uses from dags.<module> import … in a Python DAG under astro dev pytest --standalone. Live scheduler loads the DAG; astro dev pytest --standalone (DagBag scan) passes

Standalone mode previously set PYTHONPATH=AIRFLOW_HOME/include only, which
diverged from the Astro runtime Docker image. The runtime image sets
PYTHONPATH=AIRFLOW_HOME (see astro-runtime image/Dockerfile), making every
subdirectory of the project root importable as a namespace package — e.g.
`from include.utils import x` and `from dags.my_blueprints import Extract`.

Because standalone only had include/ on PYTHONPATH, imports that work in
production (and pass `astro deploy --parse`) failed when the same project
was tested locally with `astro dev start --standalone` or
`astro dev pytest --standalone`.

This change aligns standalone with production by setting PYTHONPATH to
AIRFLOW_HOME. Existing `from include.<module> import …` imports continue
to work (include/ is still findable as a namespace package under
AIRFLOW_HOME). Users who relied on flat `import <module>` from include/
will need to switch to the prefixed form, which is what the official
docs already teach (https://www.astronomer.io/docs/learn/airflow-importing-custom-hooks-operators).
@jeremybeard jeremybeard changed the title Standalone: set PYTHONPATH=AIRFLOW_HOME to match the production image Standalone: set PYTHONPATH=AIRFLOW_HOME to match runtime image May 22, 2026
@coveralls-official
Copy link
Copy Markdown

Coverage Report for CI Build 8

Coverage increased (+0.002%) to 39.653%

Details

  • Coverage increased (+0.002%) from the base build.
  • Patch coverage: 1 uncovered change across 1 file (6 of 7 lines covered, 85.71%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
airflow/standalone.go 7 6 85.71%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 66078
Covered Lines: 26202
Line Coverage: 39.65%
Coverage Strength: 9.38 hits per line

💛 - Coveralls

@jeremybeard jeremybeard marked this pull request as ready for review May 22, 2026 14:50
@jeremybeard jeremybeard requested a review from a team as a code owner May 22, 2026 14:50
@jeremybeard jeremybeard merged commit 6468bfb into main May 22, 2026
6 of 7 checks passed
@jeremybeard jeremybeard deleted the fix-standalone-pythonpath branch May 22, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants