Skip to content

Extend DEFAULT_SENSITIVE_FIELDS with webhook_url, bearer, dsn, auth_header, service_key#66673

Merged
vatsrahul1001 merged 1 commit into
apache:mainfrom
potiuk:extend-default-sensitive-fields
May 15, 2026
Merged

Extend DEFAULT_SENSITIVE_FIELDS with webhook_url, bearer, dsn, auth_header, service_key#66673
vatsrahul1001 merged 1 commit into
apache:mainfrom
potiuk:extend-default-sensitive-fields

Conversation

@potiuk
Copy link
Copy Markdown
Member

@potiuk potiuk commented May 10, 2026

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker for
masking Variables and Connection extras. Several common field names used by
official Airflow providers and standard HTTP/database configurations are
not in the allowlist.

This PR adds five field names commonly used in connection extras and
provider configurations:

  • webhook_url — Slack provider webhook URL key
  • bearer — HTTP bearer-token auth key
  • dsn — database connection strings (which typically embed credentials, e.g. postgres://user:pass@host/db)
  • auth_header — custom HTTP auth header values
  • service_key — service-account-like keys

The matcher uses case-insensitive substring matching, so e.g. bearer covers
Bearer, bearer_token, auth_bearer, etc.

Related: https://github.com/airflow-s/airflow-s/issues/377

Test plan

  • uv run --project shared/secrets_masker pytest shared/secrets_masker/tests/ -xvs — secrets-masker tests pass
  • New parametrised test asserts should_hide_value_for_key(name) returns True for each of the five new field names plus substring variants (WEBHOOK_URL, slack_webhook_url, auth_bearer, AUTH_HEADER, custom_auth_header, my_service_key)
Was generative AI tooling used to co-author this PR?
  • Yes — Claude Opus 4.7 (1M context)

Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377

Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
@potiuk potiuk requested review from amoghrajesh and ashb as code owners May 10, 2026 22:11
@potiuk potiuk added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label May 10, 2026
@potiuk potiuk added this to the Airflow 3.2.2 milestone May 10, 2026
@vatsrahul1001 vatsrahul1001 added the ready for maintainer review Set after triaging when all criteria pass. label May 12, 2026
Copy link
Copy Markdown
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this.

One thing worth considering is that providers already declare which extra fields are sensitive via "format": "password" in provider.yaml conn-fieldsand ProvidersManager already tracks this as is_sensitive=True on each ConnectionFormWidgetInfo. If we can get the secrets masker to read that and add it in, this whole process could be automated.

That way any provider that correctly declares a password field gets it masked automatically, without needing manual additions to the core allowlist.

@vatsrahul1001
Copy link
Copy Markdown
Contributor

Thanks for adding this.

One thing worth considering is that providers already declare which extra fields are sensitive via "format": "password" in provider.yaml conn-fieldsand ProvidersManager already tracks this as is_sensitive=True on each ConnectionFormWidgetInfo. If we can get the secrets masker to read that and add it in, this whole process could be automated.

That way any provider that correctly declares a password field gets it masked automatically, without needing manual additions to the core allowlist.

This can be in follow PR right @amoghrajesh ?

Copy link
Copy Markdown
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally, rest for follow up

@vatsrahul1001 vatsrahul1001 merged commit 32ac8ad into apache:main May 15, 2026
78 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-2-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request May 15, 2026
…eld names (apache#66673)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
aws-airflow-bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request May 15, 2026
…eld names (apache#66673)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
Nishieee pushed a commit to Nishieee/airflow that referenced this pull request May 15, 2026
…pache#66673)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: https://github.com/airflow-s/airflow-s/issues/377

Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
shahar1 pushed a commit that referenced this pull request May 15, 2026
…eld names (#66673) (#66991)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: https://github.com/airflow-s/airflow-s/issues/377
(cherry picked from commit 32ac8ad)


Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 pushed a commit that referenced this pull request May 20, 2026
…eld names (#66673) (#66991)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)


Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 pushed a commit that referenced this pull request May 20, 2026
…eld names (#66673) (#66991)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)


Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 pushed a commit that referenced this pull request May 21, 2026
…eld names (#66673) (#66991)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)


Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants