Skip to content

Secrets masker: non-string values are not redacted when the key name is sensitive #71275

Description

@wilsonjefferson

Under which category would you file this issue?

Airflow Core

Apache Airflow version

3.3.0

What happened and how to reproduce it?

Issue Description

When a Variable's key name matches the sensitive-keyword list, GET /api/v2/variables
correctly returns *** for string values, but returns non-string values (int, float,
bool) in cleartext.

SecretsMasker._redact_all is the fail-closed path used when a key name is judged
sensitive, but it only replaces str, recurses into containers, and returns every
other type unchanged:

# airflow/_shared/secrets_masker/secrets_masker.py:323-347
def _redact_all(self, item, depth, max_depth=MAX_RECURSION_DEPTH, *, replacement="***"):
    if depth > max_depth or isinstance(item, str):
        return replacement
    if isinstance(item, dict): ...
    if isinstance(item, (tuple, set)): ...
    if isinstance(item, list): ...
    return item          # <-- non-str scalars pass through unredacted

It is reached from _redact line 358:

if name and self.should_hide_value_for_key(name):
    return self._redact_all(item, depth, max_depth, replacement=replacement)

The comment at lines 353-355 states that key-name-based redaction "must fail closed at
any nesting level". It fails closed on depth, but not on type.

Steps to reproduce

  1. Create Variables whose names contain a sensitive keyword:
airflow variables set test-password-a "abcd"
airflow variables set test-password-b "1234"
airflow variables set test-password-c "12ab"
airflow variables set test-password-d "1234.5"
airflow variables set test-password-e "true"
  1. Read them back via the API (or view Admin -> Variables in the UI):
curl -H "Authorization: Bearer $TOKEN" \
  "http://<airflow>/api/v2/variables?limit=100"
  1. Observed:
value returned
abcd ***
12ab ***
1234 1234
1234.5 1234.5
true true

Values that are valid JSON scalars (int, float, bool) are returned unmasked; values
that remain strings are masked correctly.

Not affected: task logs

Task logs mask correctly for numeric values, because that path registers the secret
value as a substring pattern rather than going through _redact_all:

@task
def leak_test():
    from airflow.sdk import Variable
    print("numeric:", Variable.get("test-password-num"))   # 12345678 -> ***
    print("string:",  Variable.get("test-password-long"))  # abcdefgh -> ***

Both are masked. (Use values of 5+ characters — shorter ones hit the
"Skipping masking for a secret as it's too short (<5 chars)" guard.)

The same code exists at the same line numbers in
airflow/sdk/_shared/secrets_masker/secrets_masker.py.

What you think should happen instead?

Once should_hide_value_for_key(name) returns True, the value should be redacted
regardless of its Python type. A numeric PIN, account number, or all-digit API key
stored under a *_password / *_token key is a realistic case, and the current
behaviour silently returns it in cleartext while an equivalent alphanumeric value
is masked.

A possible fix is to invert the type check in _redact_all so that containers are
traversed and everything else is replaced:

if depth > max_depth or not isinstance(item, (dict, tuple, set, list)):
    return replacement

I'm not sure whether non-strings are deliberately preserved for merge(), which
restores original values where *** is unchanged — a maintainer should confirm
that before applying anything like the above.

Operating System

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian

Deployment

Docker-Compose

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

No response

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

Based on apache/airflow:3.3.0, with apache-airflow-providers-keycloak==0.8.2 installed. No changes to core Airflow.

Anything else?

ccurs every time. Deployment uses the Keycloak auth manager, but the code path is in core and is not auth-manager-specific. Python 3.13.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions