Skip to content

Set the JWTGenerator private_key field to None for log retrieval#54442

Merged
ashb merged 1 commit into
apache:mainfrom
ianbuss:do-not-load-pk-file-task-handler
Aug 13, 2025
Merged

Set the JWTGenerator private_key field to None for log retrieval#54442
ashb merged 1 commit into
apache:mainfrom
ianbuss:do-not-load-pk-file-task-handler

Conversation

@ianbuss
Copy link
Copy Markdown
Contributor

@ianbuss ianbuss commented Aug 13, 2025

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified

The fix is to add back the explicit private_key=None, for which we have to add a mypy type ignore comment as it does not seem to understand that None is allowed for AllowedPrivateKeys | None.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@ashb ashb merged commit 4fa0d01 into apache:main Aug 13, 2025
59 checks passed
github-actions Bot pushed a commit that referenced this pull request Aug 13, 2025
… for service logs retrieval (#54442)

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

```
[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified
```

The fix is to add back the explicit `private_key=None`, for which we have to add a mypy type ignore comment as it does not seem to understand that `None` is allowed for `AllowedPrivateKeys | None`.
(cherry picked from commit 4fa0d01)

Co-authored-by: Ian Buss <ianbuss@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

Status Branch Result
v3-0-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 13, 2025
… for service logs retrieval (apache#54442)

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

```
[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified
```

The fix is to add back the explicit `private_key=None`, for which we have to add a mypy type ignore comment as it does not seem to understand that `None` is allowed for `AllowedPrivateKeys | None`.
(cherry picked from commit 4fa0d01)

Co-authored-by: Ian Buss <ianbuss@users.noreply.github.com>
ashb pushed a commit that referenced this pull request Aug 13, 2025
… for service logs retrieval (#54442) (#54444)

In a recent change, the private_key parameter is left empty when instantiating the JWTGenerator used to authenticate log serving requests for in-progress tasks. Such requests use the shared secret_key instead. However, leaving private_key unset means that it is automatically populated through the attrs field factory and this leads to a failure in the post init which asserts mutual exclusivity of secret_key and private_key. The result is stack traces in the API Server logs like the following when retrieving "live" logs:

```
[2025-08-12T19:33:59.403+0000] {file_task_handler.py:907} ERROR - Could not read served logs
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 878, in _read_from_logs_server
    response = _fetch_logs_from_service(url, rel_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/airflow/utils/log/file_task_handler.py", line 173, in _fetch_logs_from_service
    generator = JWTGenerator(
                ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified
```

The fix is to add back the explicit `private_key=None`, for which we have to add a mypy type ignore comment as it does not seem to understand that `None` is allowed for `AllowedPrivateKeys | None`.
(cherry picked from commit 4fa0d01)

Co-authored-by: Ian Buss <ianbuss@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants