Skip to content

Document EdgeExecutor migration from internal_api_secret_key to jwt_secret #51905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

bdsoha
Copy link
Contributor

@bdsoha bdsoha commented Jun 18, 2025

This PR updated the documentation to use the new [api_auth] jwt_secret required configuration for EdgeExecutor instead of the old internal_api_secret_key.

Without it, the following error is emitted:

 ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
   ____   __           _      __         __
  / __/__/ /__ ____   | | /| / /__  ____/ /_____ ____
 / _// _  / _ `/ -_)  | |/ |/ / _ \/ __/  '_/ -_) __/
/___/\_,_/\_, /\__/   |__/|__/\___/_/ /_/\_\\__/_/
         /___/
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py", line 55, in main
    args.func(args)
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line 112, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py", line 55, in wrapped_function
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/edge3/cli/edge_command.py", line 522, in worker
    run_command_with_daemon_option(
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/daemon_utils.py", line 86, in run_command_with_daemon_option
    callback()
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/edge3/cli/edge_command.py", line 525, in <lambda>
    callback=lambda: _launch_worker(args),
                     ^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py", line 55, in wrapped_function
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/edge3/cli/edge_command.py", line 513, in _launch_worker
    edge_worker.start()
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/edge3/cli/edge_command.py", line 337, in start
    self.last_hb = worker_register(
                   ^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/edge3/cli/api_client.py", line 119, in worker_register
    result = _make_generic_request(
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 338, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 477, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 378, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 400, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 480, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/edge3/cli/api_client.py", line 91, in _make_generic_request
    generator = jwt_generator()
                ^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/edge3/cli/api_client.py", line 85, in jwt_generator
    return JWTGenerator(
           ^^^^^^^^^^^^^
  File "<attrs generated methods airflow.api_fastapi.auth.tokens.JWTGenerator>", line 34, in __init__
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/tokens.py", line 417, in __attrs_post_init__
    raise ValueError("Exactly one of private_key and secret_key must be specified")
ValueError: Exactly one of private_key and secret_key must be specified

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@bdsoha bdsoha requested a review from jscheffl as a code owner June 18, 2025 20:47
@bdsoha bdsoha force-pushed the feature/docs-edge-executor branch from c98e78c to a57cf5a Compare June 18, 2025 20:50
@bdsoha bdsoha changed the title Document the need for a edge workers to have a matching JWT secret Document EdgeExecutor migration from internal_api_secret_key to jwt_secret Jun 19, 2025
@bdsoha bdsoha force-pushed the feature/docs-edge-executor branch from d55e0c2 to 7115bd2 Compare June 19, 2025 02:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant