Skip to content

feat: per-component enableBuiltInSecretEnvVars for workers and scheduler#70053

Open
anmolxlight wants to merge 1 commit into
apache:mainfrom
anmolxlight:fix-70008-hide-sensitive-env-vars
Open

feat: per-component enableBuiltInSecretEnvVars for workers and scheduler#70053
anmolxlight wants to merge 1 commit into
apache:mainfrom
anmolxlight:fix-70008-hide-sensitive-env-vars

Conversation

@anmolxlight

Copy link
Copy Markdown
Contributor

Motivation

The Helm chart currently injects all sensitive config values (database connection, broker URL, fernet key, api secret key, celery result backend) into every component's environment via standard_airflow_environment. For worker pods that execute untrusted DAG code, this means user tasks can read these secrets with printenv or /proc/self/environ.

The existing global enableBuiltInSecretEnvVars flag lets operators disable these env vars, but it disables them for all components (scheduler, api-server, triggerer, worker, etc.), which may break control-plane components that legitimately need them.

What this PR does

  • Adds workers.celery.enableBuiltInSecretEnvVars (type: object | null, default: null)
  • Adds scheduler.enableBuiltInSecretEnvVars (type: object | null, default: null)
  • When a key is set in a component-level override, it takes precedence over the global value for that component only
  • When null or a key is absent, the global enableBuiltInSecretEnvVars is used (backward compatible)

Usage

workers:
  celery:
    enableBuiltInSecretEnvVars:
      AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: false
      AIRFLOW__CELERY__BROKER_URL: false

This disables the database connection and broker URL env vars on worker containers only, while other components continue to receive them from the global setting.

Implementation

The standard_airflow_environment template accepts an optional envOverrides dict. Each component call site passes its component-level map when available. The template resolves each env var by checking the component override first, then falling back to the global.

Backward Compatibility

Fully backward compatible. Default values (null) produce identical behavior to the existing global-only approach.

Closes #70008

Adds workers.celery.enableBuiltInSecretEnvVars and scheduler.enableBuiltInSecretEnvVars
to override the global enableBuiltInSecretEnvVars map on a per-component basis.

This allows operators to disable sensitive env vars (e.g. database conn string,
broker URL, fernet key) on worker containers where untrusted DAG code executes,
while keeping them on control-plane components.

The standard_airflow_environment template accepts an optional envOverrides dict.
Each component call site passes its component-level map. The template resolves
each env var by checking the component override first, then falling back to
the global enableBuiltInSecretEnvVars.

Closes apache#70008
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:helm-chart Airflow Helm Chart

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hiding sensitive environment variables

1 participant