Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Permitting airflow kerberos to run in different modes #35146

Merged
merged 12 commits into from
Oct 25, 2023
Merged

Permitting airflow kerberos to run in different modes #35146

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
9344faa
Allowing airflow kerberos mode to be configurable
amoghrajesh Oct 24, 2023
baefbe8
moving the config to flag instead
amoghrajesh Oct 24, 2023
5d3ec59
removing config.yml changes
amoghrajesh Oct 24, 2023
641e73d
making code robust
amoghrajesh Oct 24, 2023
ff763f4
updating docs
amoghrajesh Oct 24, 2023
5824eb3
Merge branch 'main' into kerberosMode
amoghrajesh Oct 25, 2023
b11501a
addressing jed and potiuks comments
amoghrajesh Oct 25, 2023
a12efbc
fixing docs
amoghrajesh Oct 25, 2023
643554e
extra line removed
amoghrajesh Oct 25, 2023
8baa3f9
fixing test text
amoghrajesh Oct 25, 2023
e13fb38
fixing docs and tests
amoghrajesh Oct 25, 2023
900c16a
fixing docs and tests
amoghrajesh Oct 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 8 additions & 0 deletions airflow/config_templates/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2350,6 +2350,14 @@ triggerer:
kerberos:
description: ~
options:
mode:
amoghrajesh marked this conversation as resolved.
Show resolved Hide resolved
description: |
Specify the mode in which kerberos container will run, can be sidecar mode vs init container mode.
The sidecar mode runs indefinitely but the init container mode performs kinit once and exits.
version_added: 2.7.3
amoghrajesh marked this conversation as resolved.
Show resolved Hide resolved
type: string
example: ~
default: "sidecar"
ccache:
description: |
Location of your ccache file once kinit has been performed.
Expand Down
16 changes: 15 additions & 1 deletion airflow/security/kerberos.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@

log = logging.getLogger(__name__)

SIDECAR_MODE = "sidecar"
INIT_MODE = "init"
DEFAULT_MODE = SIDECAR_MODE


def get_kerberos_principle(principal: str | None) -> str:
"""Retrieve Kerberos principal. Fallback to principal from Airflow configuration if not provided."""
Expand Down Expand Up @@ -188,6 +192,16 @@ def run(principal: str | None, keytab: str):
log.warning("Keytab renewer not starting, no keytab configured")
sys.exit(0)

while True:
mode = conf.get("kerberos", "mode")
if mode != INIT_MODE or mode != SIDECAR_MODE:
mode = DEFAULT_MODE

log.info("Using airflow kerberos mode: %s", mode)

if mode == SIDECAR_MODE:
while True:
renew_from_kt(principal, keytab)
time.sleep(conf.getint("kerberos", "reinit_frequency"))
else:
renew_from_kt(principal, keytab)
time.sleep(conf.getint("kerberos", "reinit_frequency"))
amoghrajesh marked this conversation as resolved.
Show resolved Hide resolved
6 changes: 6 additions & 0 deletions docs/apache-airflow/security/kerberos.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,12 @@ Launch the ticket renewer by
# run ticket renewer
airflow kerberos

To support more advanced deployment models for using kerberos in an init fashion or sidecar fashion, you can specify the mode
field in your config.yml, specify mode as either "init" or "sidecar".

* Sidecar: The airflow kerberos process will run forever
* Init: The airflow kerberos will run once and exit. In case of failure the main container won't spin up.

Hadoop
^^^^^^

Expand Down