Skip to content

Fix Airflow 3 task log access with NetworkPolicies#65754

Merged
potiuk merged 1 commit into
apache:mainfrom
henry3260:airflow3-log-networkpolicy-api-server
Apr 28, 2026
Merged

Fix Airflow 3 task log access with NetworkPolicies#65754
potiuk merged 1 commit into
apache:mainfrom
henry3260:airflow3-log-networkpolicy-api-server

Conversation

@henry3260
Copy link
Copy Markdown
Contributor

Why

In Airflow 3, task logs are fetched through the API server rather than the webserver.

The Helm chart still allowed ingress to the scheduler, triggerer, and worker log-serving ports only from pods labeled component: webserver. When networkPolicies.enabled=true, this can block the API server from reaching those log endpoints and break task log access in Airflow 3.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added the area:helm-chart Airflow Helm Chart label Apr 24, 2026
@henry3260 henry3260 force-pushed the airflow3-log-networkpolicy-api-server branch from 7f95efc to e43e20d Compare April 24, 2026 08:32
Copy link
Copy Markdown
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Some second maintainer review?

@jscheffl jscheffl added the backport-to-chart/v1-2x-test Automatic backport to chart 1.2x maintenance branch label Apr 24, 2026
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Apr 27, 2026
Copy link
Copy Markdown
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but needs rebase.

@henry3260 henry3260 force-pushed the airflow3-log-networkpolicy-api-server branch from e43e20d to c5fb422 Compare April 28, 2026 05:51
@henry3260 henry3260 requested a review from dstandish as a code owner April 28, 2026 05:51
@henry3260 henry3260 requested a review from potiuk April 28, 2026 07:08
@potiuk potiuk merged commit 8b2ce00 into apache:main Apr 28, 2026
104 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

Backport failed to create: chart/v1-2x-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
chart/v1-2x-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 8b2ce00 chart/v1-2x-test

This should apply the commit to the chart/v1-2x-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@jscheffl
Copy link
Copy Markdown
Contributor

@potiuk Are you adding the backport?

@henry3260 henry3260 mentioned this pull request Apr 29, 2026
1 task
potiuk added a commit that referenced this pull request May 31, 2026
…67805)

(cherry picked from commit 8b2ce00)

Co-authored-by: Henry Chen <henryhenry0512@gmail.com>
potiuk added a commit that referenced this pull request May 31, 2026
…67817)

The 1.2x chart line supports both Airflow 2.11 (logs served by the webserver) and Airflow 3 (logs served by the api-server). #65754 unconditionally changed the scheduler/triggerer/worker NetworkPolicy log-ingress selectors to api-server, which would break task log access on Airflow 2.11 deployments. Choose the component based on .Values.airflowVersion via semverCompare (>=3.0.0 -> api-server, else webserver), matching how the chart already selects webserver vs api-server elsewhere. Tests now cover both Airflow versions.
potiuk added a commit that referenced this pull request May 31, 2026
…67819)

With #67817 the NetworkPolicy log-access fix (#65754) is version-aware and works for both Airflow 2.11 and 3, so reword the 1.22.0 changelog entry (and matching ArtifactHub annotation) from 'Airflow 3' to 'Airflow 2 and 3'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:helm-chart Airflow Helm Chart backport-to-chart/v1-2x-test Automatic backport to chart 1.2x maintenance branch ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants