Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions airflow-core/src/airflow/cli/commands/team_command.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,16 @@ def _show_teams(teams, output):
)


TEAM_NAME_PATTERN = r"^[a-zA-Z0-9_-]{3,50}$"


def _extract_team_name(args):
"""Extract and validate team name from args."""
team_name = args.name.strip()
if not team_name:
raise SystemExit("Team name cannot be empty")
if not re.match(r"^[a-zA-Z0-9_-]{3,50}$", team_name):
raise SystemExit("Invalid team name: must match regex ^[a-zA-Z0-9_-]{3,50}$")
if not re.match(TEAM_NAME_PATTERN, team_name):
raise SystemExit(f"Invalid team name: must match regex {TEAM_NAME_PATTERN}")
return team_name


Expand Down Expand Up @@ -169,6 +172,18 @@ def team_sync(args, *, session=NEW_SESSION):
if bundle.team_name is not None
}

# The bundle config is a second creation path for teams, so it has to enforce the same
# name rule as `teams create`. Other code relies on a stored team name being a valid one
# -- the environment secrets backend decides whether a supplied secret id could name a
# team namespace by testing the leading segment against this pattern, and an unvalidated
# short name such as "a" would make that test miss.
invalid = sorted(name for name in dag_bundle_teams if not re.match(TEAM_NAME_PATTERN, name))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This validates the names in the Dag bundle config, but the secrets guard depends on every stored team name being valid — and teams sync shipped in 3.3.0 with no validation at all. A live 3.3.0 deployment can already hold a team named ab, and after the companion change _ab___x resolves through the team agnostic lookup into AIRFLOW_CONN__AB___X, that team's secret. That's the read #70736 closed, and _ab___x is one of the two ids the new test asserts must resolve.

The config-side check catches this only while the short name is still in the config; a team row created by an earlier sync and since dropped from the config is missed. The stored names are in hand one line later (Team.get_all_team_names(session=session)), so covering them too is nearly free — hard error, or a warning if failing on pre-existing data is too aggressive for a patch release.

Related, given the backport-to-v3-3-test label: on a 3.3.x patch upgrade, a deployment whose bundle config carries a short team name will see airflow teams sync start exiting non-zero where it previously succeeded. Defensible, but better as a deliberate call than a side effect.

Separately — re.match with a $-anchored pattern accepts a trailing newline, and unlike teams create this path has no .strip(). See the note on _TEAM_NAME in environment_variables.py.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Answering the question directly, since you asked for it to be a deliberate call rather than a side effect: hard error, and no backport.

This PR is now a draft on 3.4.0 with backport-to-v3-3-test removed, and the work moved to #70886. So the "3.3.x patch upgrade starts exiting non-zero" case you flagged does not arise — nothing here ships in a patch release. Given that, the warning option loses its only real justification, and a warning would leave the deployment believing its teams are isolated when they are not.

Both halves are implemented in #70886: teams sync validates the stored names alongside the bundle-config ones (Team.get_all_team_names as you pointed out, one line down and nearly free), and it exits naming the offending names.

Your other two points are in there as well — one unanchored pattern used with re.fullmatch at both sites, defined once in the secrets backend and imported by the CLI, with test_team_name_pattern_is_shared_with_the_secrets_backend so drift fails CI rather than silently reopening the hole. You were right about re.match and the trailing newline; my first attempt at that test asserted teams create rejects "team1\n" and it does not, because _extract_team_name strips first. The exposure is on the sync path exactly as you said, and the test moved there.

And the finding at the top of this review — the bare-id spelling — I reproduced it. A caller in team_a asking for prod___dbconn returns team_a___prod's secret. It is the reason this PR is held rather than fixed up: narrowing the team agnostic refusal cannot reach a read that the scoped lookup satisfies. #70886 closes it at the source by making team_a___prod unspellable, relaxed from forbidding every underscore to forbidding consecutive ones so team_a stays valid.

Worth being explicit about one consequence: this PR's own new test asserted _ab___x must resolve, and with an unvalidated stored team named ab that assertion pins the vulnerable behaviour. That is what your comment caught.

if invalid:
raise SystemExit(
f"Invalid team name(s) in the dag bundle config: {', '.join(invalid)}. "
f"Team names must match regex {TEAM_NAME_PATTERN}."
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No test covers this new behaviour — the diff touches only test_secrets_environment_variables.py, and test_team_sync is unchanged and happy-path only.

Flag any changed or added behaviour without a corresponding test, and flag tests that a reviewer cannot fail by reverting the PR's change. The target is exactly 100% coverage of what the PR changes — no more, no less.

.github/instructions/code-review.instructions.md § Testing Requirements

The PR body's "21 cases pass with the shared pattern" confirms nothing regressed, which isn't the same thing: delete this if invalid: block and the suite stays green. Since this guard is what makes the secrets relaxation sound, it's the behaviour here that most needs pinning — one case with "team_name": "a" in the bundle config asserting SystemExit and that no team row is created.


teams_added = 0

try:
Expand Down
54 changes: 37 additions & 17 deletions airflow-core/src/airflow/secrets/environment_variables.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,15 @@
CONN_ENV_PREFIX = "AIRFLOW_CONN_"
VAR_ENV_PREFIX = "AIRFLOW_VAR_"

# Separator between the team name and the secret id in a team namespaced
# environment variable name: AIRFLOW_CONN__<TEAM>___<ID>.
TEAM_SEP = "___"

# What ``airflow teams create`` accepts as a team name, so what a team
# namespace can actually be spelled with.
# Kept in sync with ``airflow.cli.commands.team_command``.
_TEAM_NAME = re.compile(r"[a-zA-Z0-9_-]{3,50}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two spellings aren't equivalent, which undercuts the "kept in sync" promise: $ also matches before a trailing newline, so

re.match(r"^[a-zA-Z0-9_-]{3,50}$", "team1\n")   # match
_TEAM_NAME.fullmatch("team1\n")                  # None

(verified). teams create is saved by its .strip(); the new teams sync path has none, so a Dag bundle config with a trailing newline in team_name creates a team this guard cannot recognise. One unanchored pattern plus re.fullmatch at both sites removes the divergence entirely.

Also: a # Kept in sync with ... comment is the kind of promise better held by a one-line test asserting the two constants are equal — otherwise drift silently reopens the hole instead of failing CI.

Minor: the comment above now understates the coupling — after this PR the pattern is what teams create and teams sync accept.



class EnvironmentVariablesBackend(BaseSecretsBackend):
"""Retrieves Connection object and Variable from environment variable."""
Expand Down Expand Up @@ -65,25 +74,36 @@ def get_variable(self, key: str, team_name: str | None = None) -> str | None:
@staticmethod
def _names_a_team_namespace(secret_id: str) -> bool:
"""
Whether ``secret_id`` spells out a team namespaced environment variable name.
Whether ``secret_id`` could spell out a team namespaced environment variable name.

A team specific secret lives in the ``_<TEAM_NAME>___<SECRET_ID>`` namespace of the
environment. An id of that shape therefore makes the team agnostic lookup -- which
prepends only ``AIRFLOW_CONN_`` / ``AIRFLOW_VAR_`` -- land inside some team's namespace,
so that lookup is refused for such an id.
environment. An id of that shape makes the team agnostic lookup -- which prepends only
``AIRFLOW_CONN_`` / ``AIRFLOW_VAR_`` -- land inside some team's namespace, so that
lookup is refused for such an id.

The test is whether the leading segment **could be a team name**, not merely whether
the id contains the separator. Team names are validated on creation, so an id like
``_a___b`` cannot name a team namespace -- ``a`` is too short to be a team -- and
refusing it would block a legitimate team agnostic secret for no benefit. Every id
that does spell a real team's namespace has a valid team name in that position by
construction, so nothing reachable is let through.

**The id is never attributed to a particular team**, because it cannot be: a team name
may itself contain the ``___`` separator, so ``_a___b___c`` is both team ``a`` with id
``b___c`` and team ``a___b`` with id ``c``, and nothing in the string distinguishes them.
Only the caller's own namespace is ever constructed, never parsed.

This is why the check is not "does the id belong to some team other than the caller's".
Comparing the id against the prefix the caller's team builds looks equivalent and is not:
for a caller in team ``a`` the id ``_a___b___c`` starts with ``_A___``, yet the variable
it resolves, ``AIRFLOW_CONN__A___B___C``, is team ``a___b``'s. Treating a prefix match as
ownership hands one team the secrets of every team whose name extends it. Callers reach
their own team's secrets with the bare id plus their team scope -- handled above, and safe
because that path can only ever build the caller's own namespace -- so nothing legitimate
needs a namespaced id here.
may itself contain the separator, so ``_a___b___c`` is both team ``a`` with id
``b___c`` and team ``a___b`` with id ``c``. Every split is therefore considered, and
one plausible team name is enough to refuse. Comparing the id against the prefix the
caller's own team builds looks equivalent and is not: for a caller in team ``a`` the
id ``_a___b___c`` starts with ``_A___``, yet the variable it resolves,
``AIRFLOW_CONN__A___B___C``, is team ``a___b``'s. Treating a prefix match as ownership
hands one team the secrets of every team whose name extends it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleting the old "that path can only ever build the caller's own namespace" sentence is the right call — it was false. But the gap it papered over is still open, and it's the same cross-team read #70736 closed, just spelled differently.

A caller in team_a supplying the bare id prod___dbconn never reaches this guard: the scoped branch above returns first, and the id has no leading _ anyway.

# caller: get_conn_value("prod___dbconn", team_name="team_a")
f"{CONN_ENV_PREFIX}_{team_name.upper()}___" + conn_id.upper()
# -> AIRFLOW_CONN__TEAM_A___PROD___DBCONN

# team "team_a___prod" stores "dbconn" as
# "AIRFLOW_CONN_" + "_TEAM_A___PROD" + "___" + "DBCONN"
# -> AIRFLOW_CONN__TEAM_A___PROD___DBCONN     <- identical

I checked; the strings are byte-identical. test_team_whose_name_extends_the_callers_is_not_readable uses this exact pair of team names but only the namespaced spelling.

Pre-existing rather than introduced here, so not a blocker on this diff — but worth a follow-up, and the root cause is shared with the loop below: ___ isn't reserved, so _<TEAM>___<ID> doesn't parse uniquely. Forbidding ___ inside a team name would make it unambiguous, collapse this loop into one exact split, and close the scoped-path collision too.

"""
return re.fullmatch(r"_.+___.+", secret_id) is not None
if not secret_id.startswith("_"):
return False
# Overlapping positions matter: ``_abc____x`` splits at both index 4 and 5.
for i in range(1, len(secret_id) - len(TEAM_SEP) + 1):
if secret_id[i : i + len(TEAM_SEP)] != TEAM_SEP:
continue
team, rest = secret_id[1:i], secret_id[i + len(TEAM_SEP) :]
if rest and _TEAM_NAME.fullmatch(team):
return True
return False
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,22 @@ def test_id_spelling_out_a_team_namespace_is_never_resolved(

assert lookup(env_prefix, method, team_scoped_id(team_name), team_name) is None

@pytest.mark.parametrize(("env_prefix", "method"), LOOKUPS)
@pytest.mark.parametrize("secret_id", ["_a___b", "_ab___x"])
@pytest.mark.parametrize("team_name", [None, *TEAM_NAMES])
def test_id_that_cannot_name_a_team_is_still_resolved(
self, monkeypatch, env_prefix, method, secret_id, team_name
):
"""Refusing every ``_x___y`` id would block legitimate team agnostic secrets.

Team names are validated as ``^[a-zA-Z0-9_-]{3,50}$``, so a one- or two-character
leading segment cannot be a team. Such an id names no team's namespace and must stay
resolvable through the team agnostic lookup, for any caller.
"""
monkeypatch.setenv(env_prefix + secret_id.upper(), GLOBAL_VALUE)

assert lookup(env_prefix, method, secret_id, team_name) == GLOBAL_VALUE

@pytest.mark.parametrize(("env_prefix", "method"), LOOKUPS)
def test_team_whose_name_extends_the_callers_is_not_readable(self, monkeypatch, env_prefix, method):
"""A team name may contain the ``___`` separator, so one team's namespace can start with another's.
Expand Down