Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix leak sensitive field via V1EnvVar on exception #29016

Merged
merged 9 commits into from Feb 23, 2023

Commits on Feb 18, 2023

  1. Fix leak sensitive field via V1EnvVar on exception

    Currently the KubernetesPodOperator `env_vars` will be printed
    on the task logs if there is any templating error (like an
    `UndefinedError`, `TemplateSyntaxError` or `KeyError`)
    
    ```
    [2023-01-16, 23:03:17 UTC] {abstractoperator.py:592} ERROR - Exception rendering Jinja template for task 'dry_run_demo', field 'env_vars'. Template: [{'name': 'password', 'value': 'secretpassword', 'value_from': None}, {'name': 'VAR2', 'value': '{{ var.value.nonexisting}}', 'value_from': None}]
    Traceback (most recent call last):
      File "/Users/rubelagu/.pyenv/versions/3.10.7/envs/venv-airflow-250/lib/python3.10/site-packages/airflow/models/abstractoperator.py", line 585, in _do_render_template_fields
        rendered_content = self.render_template(
      File "/Users/rubelagu/.pyenv/versions/3.10.7/envs/venv-airflow-250/lib/python3.10/site-packages/airflow/models/abstractoperator.py", line 657, in render_template
        return [self.render_template(element, context, jinja_env, oids) for element in value]
      File "/Users/rubelagu/.pyenv/versions/3.10.7/envs/venv-airflow-250/lib/python3.10/site-packages/airflow/models/abstractoperator.py", line 657, in <listcomp>
        return [self.render_template(element, context, jinja_env, oids) for element in value]
      File "/Users/rubelagu/.pyenv/versions/3.10.7/envs/venv-airflow-250/lib/python3.10/site-packages/airflow/models/abstractoperator.py", line 664, in render_template
        self._render_nested_template_fields(value, context, jinja_env, oids)
      File "/Users/rubelagu/.pyenv/versions/3.10.7/envs/venv-airflow-250/lib/python3.10/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 321, in _render_nested_template_fields
        self._do_render_template_fields(content, ("value", "name"), context, jinja_env, seen_oids)
      ...
      ...
      File "/Users/rubelagu/.pyenv/versions/3.10.7/envs/venv-airflow-250/lib/python3.10/site-packages/airflow/models/variable.py", line 141, in get
        raise KeyError(f"Variable {key} does not exist")
    KeyError: 'Variable nonexisting does not exist'
    ```
    
    this happens when there is any error on the templates. For example
    a `KeyError` raised when using `var.value.somemistypedvalue`:
    
    ```
            env_vars={
                "password": "{{ conn.test_connection.password }}",
                "VAR2": "{{ var.value.nonexisting}}",
            },
    ```
    
    This PR uses the `airflow.utils.log.secrets_maker.redact` to remove any
    field contained in `DEFAULT_SENSITIVE_FIELDS` or
    `sensitive_var_conn_names`.
    ecerulm authored and Taragolis committed Feb 18, 2023
    Configuration menu
    Copy the full SHA
    994eecd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bf32467 View commit details
    Browse the repository at this point in the history
  3. Add comments

    ecerulm authored and Taragolis committed Feb 18, 2023
    Configuration menu
    Copy the full SHA
    d9e4733 View commit details
    Browse the repository at this point in the history
  4. Use try-import

    ecerulm authored and Taragolis committed Feb 18, 2023
    Configuration menu
    Copy the full SHA
    6da16ef View commit details
    Browse the repository at this point in the history
  5. Remove # type: ignore

    ecerulm authored and Taragolis committed Feb 18, 2023
    Configuration menu
    Copy the full SHA
    2358dba View commit details
    Browse the repository at this point in the history
  6. Fix formatting issues

    ecerulm authored and Taragolis committed Feb 18, 2023
    Configuration menu
    Copy the full SHA
    2617bb4 View commit details
    Browse the repository at this point in the history
  7. Add missing import

    ecerulm authored and Taragolis committed Feb 18, 2023
    Configuration menu
    Copy the full SHA
    1aaeacc View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2023

  1. Update kubernetes_tests/test_kubernetes_pod_operator.py

    Co-authored-by: Andrey Anshin <Andrey.Anshin@taragol.is>
    ecerulm and Taragolis committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    3ed4e8b View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2023

  1. Configuration menu
    Copy the full SHA
    859a5c8 View commit details
    Browse the repository at this point in the history