KubernetesPodOperator doesn't mask variables in Rendered Template that are used as arguments

[NikitasB]

Apache Airflow version

2.6.1

What happened

I am pulling a variable from Google Secret Manager and I'm using it as an argument in a KubernetesPodOperator task. I've also tried it with the KubernetesPodOperatorAsync operator and I'm getting the same behaviour.

The variable value is not masked on Rendered Template page. If I use the exact same variable in a different operator, like the HttpSensorAsync, it is properly masked. That is quite critical and I can't deploy the DAG to production.

What you think should happen instead

The variable in the KubernetesPodOperator should be masked and only '***' should be shown in the Rendered Template page

How to reproduce

Here's the example of code where I use the exact same variable in two different Operators. It's in the arguments of the Kubernetes Operator and then used in a different operator next.

    my_changeset = KubernetesPodOperator(
        task_id='my_load',
        namespace=kubernetes_namespace,
        service_account_name=service_account_name,
        image='my-feed:latest',
        name='changeset_load',
        in_cluster=in_cluster,
        cluster_context='docker-desktop',  # is ignored when in_cluster is set to True
        is_delete_operator_pod=True,
        get_logs=True,
        image_pull_policy=image_pull_policy,
        arguments=[
            '-k{{ var.json.faros_api_key.faros_api_key }}',
        ],
        container_resources=k8s.V1ResourceRequirements(requests=requests, limits=limits),
        volumes=volumes,
        volume_mounts=volume_mounts,
        log_events_on_failure=True,
        startup_timeout_seconds=60 * 5,
    )

    test_var = HttpSensorAsync(
        task_id=f'wait_for_my_file',
        http_conn_id='my_paymentreports_http',
        endpoint='{{ var.json.my_paymentreports_http.client_id }}/report',
        headers={'user-agent': 'King'},
        request_params={
            'access_token': '{{ var.json.faros_api_key.faros_api_key }}',
        },
        response_check=lambda response: True if response.status_code == 200 else False,
        extra_options={'check_response': False},
        timeout=60 * 60 * 8,
    )

The same {{ var.json.faros_api_key.faros_api_key }} is used in both operators, but only masked in the HttpSensorAsync operator.

Operating System

Debian GNU/Linux 11 (bullseye)

Versions of Apache Airflow Providers

apache-airflow==2.6.1+astro.3
apache-airflow-providers-amazon==8.1.0
apache-airflow-providers-celery==3.2.0
apache-airflow-providers-cncf-kubernetes==7.0.0
apache-airflow-providers-common-sql==1.5.1
apache-airflow-providers-datadog==3.3.0
apache-airflow-providers-elasticsearch==4.5.0
apache-airflow-providers-ftp==3.4.1
apache-airflow-providers-github==2.3.0
apache-airflow-providers-google==10.0.0
apache-airflow-providers-hashicorp==3.4.0
apache-airflow-providers-http==4.4.1
apache-airflow-providers-imap==3.2.1
apache-airflow-providers-microsoft-azure==6.1.1
apache-airflow-providers-mysql==5.1.0
apache-airflow-providers-postgres==5.5.0
apache-airflow-providers-redis==3.2.0
apache-airflow-providers-samba==4.2.0
apache-airflow-providers-sendgrid==3.2.0
apache-airflow-providers-sftp==4.3.0
apache-airflow-providers-slack==7.3.0
apache-airflow-providers-sqlite==3.4.1
apache-airflow-providers-ssh==3.7.0

Deployment

Astronomer

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[phanikumv]

Hi @NikitasB thank you for creating this issue. I will check it and get back to you

[phanikumv]

@NikitasB

Though you are using the same variable, you are using it differently in both the operators.

The reason it is masked in HttpSensor is because of the word access_token which is a sensitive field as per Airflow.
Please refer to below docs

https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/security/secrets/mask-sensitive-values.html#sensitive-field-names

If you want to do the same for arguments field in KPO use case, then we can do it in 2 ways. Either at DAG level or deployment level.

1.You can mask at DAG level by using a mask_secret method. Refer to

@task
def my_func():
    from airflow.utils.log.secrets_masker import mask_secret

    mask_secret("custom_value")

    ...

2.include it in the airflow config as below. (But then this change is applied across your deployment, )

[core]
sensitive_var_conn_names = arguments

Can you test out either of the approaches and let me know the outcome please

[nathadfield]

@phanikumv But the name of the variable contain the phrase api_key which is also part of the sensitive list of terms that should be masked.

How is arguments in KPO different from request_params in HTTPOperator?

[NikitasB]

Hi @phanikumv , thank you for your comment.

I tried to implement the use of masking the secret on DAG level, but that still doesn't work:

@task
def hide_git_token():
    from airflow.utils.log.secrets_masker import mask_secret
    mask_secret('{{ var.json.faros_api_key.faros_git_token }}')

[phanikumv]

@phanikumv But the name of the variable contain the phrase api_key which is also part of the sensitive list of terms that should be masked.

How is arguments in KPO different from request_params in HTTPOperator?

I believe the field name is what which matters. The field arguments in KPO task doesnt contain anything which is considered "sensitive" whereas the field request_params in HttpSensor task contains access_token

cc @potiuk @hussein-awala for confirming the above

[nathadfield]

Ok. My reading of the page on masking sensitive values suggested it was based on the name of the variable.

It will also mask the value of a Variable, or the field of a Connection’s extra JSON blob if the name contains any words in (‘access_token’, ‘api_key’, ‘apikey’,’authorization’, ‘passphrase’, ‘passwd’, ‘password’, ‘private_key’, ‘secret’, ‘token’).

If not, then I'd suggest that it needs a slight rephrasing.