-
Notifications
You must be signed in to change notification settings - Fork 17.5k
Refuse only ids whose leading segment could name a team #70884
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -48,13 +48,16 @@ def _show_teams(teams, output): | |
| ) | ||
|
|
||
|
|
||
| TEAM_NAME_PATTERN = r"^[a-zA-Z0-9_-]{3,50}$" | ||
|
|
||
|
|
||
| def _extract_team_name(args): | ||
| """Extract and validate team name from args.""" | ||
| team_name = args.name.strip() | ||
| if not team_name: | ||
| raise SystemExit("Team name cannot be empty") | ||
| if not re.match(r"^[a-zA-Z0-9_-]{3,50}$", team_name): | ||
| raise SystemExit("Invalid team name: must match regex ^[a-zA-Z0-9_-]{3,50}$") | ||
| if not re.match(TEAM_NAME_PATTERN, team_name): | ||
| raise SystemExit(f"Invalid team name: must match regex {TEAM_NAME_PATTERN}") | ||
| return team_name | ||
|
|
||
|
|
||
|
|
@@ -169,6 +172,18 @@ def team_sync(args, *, session=NEW_SESSION): | |
| if bundle.team_name is not None | ||
| } | ||
|
|
||
| # The bundle config is a second creation path for teams, so it has to enforce the same | ||
| # name rule as `teams create`. Other code relies on a stored team name being a valid one | ||
| # -- the environment secrets backend decides whether a supplied secret id could name a | ||
| # team namespace by testing the leading segment against this pattern, and an unvalidated | ||
| # short name such as "a" would make that test miss. | ||
| invalid = sorted(name for name in dag_bundle_teams if not re.match(TEAM_NAME_PATTERN, name)) | ||
| if invalid: | ||
| raise SystemExit( | ||
| f"Invalid team name(s) in the dag bundle config: {', '.join(invalid)}. " | ||
| f"Team names must match regex {TEAM_NAME_PATTERN}." | ||
| ) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No test covers this new behaviour — the diff touches only
The PR body's "21 cases pass with the shared pattern" confirms nothing regressed, which isn't the same thing: delete this |
||
|
|
||
| teams_added = 0 | ||
|
|
||
| try: | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -27,6 +27,15 @@ | |
| CONN_ENV_PREFIX = "AIRFLOW_CONN_" | ||
| VAR_ENV_PREFIX = "AIRFLOW_VAR_" | ||
|
|
||
| # Separator between the team name and the secret id in a team namespaced | ||
| # environment variable name: AIRFLOW_CONN__<TEAM>___<ID>. | ||
| TEAM_SEP = "___" | ||
|
|
||
| # What ``airflow teams create`` accepts as a team name, so what a team | ||
| # namespace can actually be spelled with. | ||
| # Kept in sync with ``airflow.cli.commands.team_command``. | ||
| _TEAM_NAME = re.compile(r"[a-zA-Z0-9_-]{3,50}") | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These two spellings aren't equivalent, which undercuts the "kept in sync" promise: re.match(r"^[a-zA-Z0-9_-]{3,50}$", "team1\n") # match
_TEAM_NAME.fullmatch("team1\n") # None(verified). Also: a Minor: the comment above now understates the coupling — after this PR the pattern is what |
||
|
|
||
|
|
||
| class EnvironmentVariablesBackend(BaseSecretsBackend): | ||
| """Retrieves Connection object and Variable from environment variable.""" | ||
|
|
@@ -65,25 +74,36 @@ def get_variable(self, key: str, team_name: str | None = None) -> str | None: | |
| @staticmethod | ||
| def _names_a_team_namespace(secret_id: str) -> bool: | ||
| """ | ||
| Whether ``secret_id`` spells out a team namespaced environment variable name. | ||
| Whether ``secret_id`` could spell out a team namespaced environment variable name. | ||
|
|
||
| A team specific secret lives in the ``_<TEAM_NAME>___<SECRET_ID>`` namespace of the | ||
| environment. An id of that shape therefore makes the team agnostic lookup -- which | ||
| prepends only ``AIRFLOW_CONN_`` / ``AIRFLOW_VAR_`` -- land inside some team's namespace, | ||
| so that lookup is refused for such an id. | ||
| environment. An id of that shape makes the team agnostic lookup -- which prepends only | ||
| ``AIRFLOW_CONN_`` / ``AIRFLOW_VAR_`` -- land inside some team's namespace, so that | ||
| lookup is refused for such an id. | ||
|
|
||
| The test is whether the leading segment **could be a team name**, not merely whether | ||
| the id contains the separator. Team names are validated on creation, so an id like | ||
| ``_a___b`` cannot name a team namespace -- ``a`` is too short to be a team -- and | ||
| refusing it would block a legitimate team agnostic secret for no benefit. Every id | ||
| that does spell a real team's namespace has a valid team name in that position by | ||
| construction, so nothing reachable is let through. | ||
|
|
||
| **The id is never attributed to a particular team**, because it cannot be: a team name | ||
| may itself contain the ``___`` separator, so ``_a___b___c`` is both team ``a`` with id | ||
| ``b___c`` and team ``a___b`` with id ``c``, and nothing in the string distinguishes them. | ||
| Only the caller's own namespace is ever constructed, never parsed. | ||
|
|
||
| This is why the check is not "does the id belong to some team other than the caller's". | ||
| Comparing the id against the prefix the caller's team builds looks equivalent and is not: | ||
| for a caller in team ``a`` the id ``_a___b___c`` starts with ``_A___``, yet the variable | ||
| it resolves, ``AIRFLOW_CONN__A___B___C``, is team ``a___b``'s. Treating a prefix match as | ||
| ownership hands one team the secrets of every team whose name extends it. Callers reach | ||
| their own team's secrets with the bare id plus their team scope -- handled above, and safe | ||
| because that path can only ever build the caller's own namespace -- so nothing legitimate | ||
| needs a namespaced id here. | ||
| may itself contain the separator, so ``_a___b___c`` is both team ``a`` with id | ||
| ``b___c`` and team ``a___b`` with id ``c``. Every split is therefore considered, and | ||
| one plausible team name is enough to refuse. Comparing the id against the prefix the | ||
| caller's own team builds looks equivalent and is not: for a caller in team ``a`` the | ||
| id ``_a___b___c`` starts with ``_A___``, yet the variable it resolves, | ||
| ``AIRFLOW_CONN__A___B___C``, is team ``a___b``'s. Treating a prefix match as ownership | ||
| hands one team the secrets of every team whose name extends it. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Deleting the old "that path can only ever build the caller's own namespace" sentence is the right call — it was false. But the gap it papered over is still open, and it's the same cross-team read #70736 closed, just spelled differently. A caller in # caller: get_conn_value("prod___dbconn", team_name="team_a")
f"{CONN_ENV_PREFIX}_{team_name.upper()}___" + conn_id.upper()
# -> AIRFLOW_CONN__TEAM_A___PROD___DBCONN
# team "team_a___prod" stores "dbconn" as
# "AIRFLOW_CONN_" + "_TEAM_A___PROD" + "___" + "DBCONN"
# -> AIRFLOW_CONN__TEAM_A___PROD___DBCONN <- identicalI checked; the strings are byte-identical. Pre-existing rather than introduced here, so not a blocker on this diff — but worth a follow-up, and the root cause is shared with the loop below: |
||
| """ | ||
| return re.fullmatch(r"_.+___.+", secret_id) is not None | ||
| if not secret_id.startswith("_"): | ||
| return False | ||
| # Overlapping positions matter: ``_abc____x`` splits at both index 4 and 5. | ||
| for i in range(1, len(secret_id) - len(TEAM_SEP) + 1): | ||
| if secret_id[i : i + len(TEAM_SEP)] != TEAM_SEP: | ||
| continue | ||
| team, rest = secret_id[1:i], secret_id[i + len(TEAM_SEP) :] | ||
| if rest and _TEAM_NAME.fullmatch(team): | ||
| return True | ||
| return False | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This validates the names in the Dag bundle config, but the secrets guard depends on every stored team name being valid — and
teams syncshipped in 3.3.0 with no validation at all. A live 3.3.0 deployment can already hold a team namedab, and after the companion change_ab___xresolves through the team agnostic lookup intoAIRFLOW_CONN__AB___X, that team's secret. That's the read #70736 closed, and_ab___xis one of the two ids the new test asserts must resolve.The config-side check catches this only while the short name is still in the config; a team row created by an earlier sync and since dropped from the config is missed. The stored names are in hand one line later (
Team.get_all_team_names(session=session)), so covering them too is nearly free — hard error, or a warning if failing on pre-existing data is too aggressive for a patch release.Related, given the
backport-to-v3-3-testlabel: on a 3.3.x patch upgrade, a deployment whose bundle config carries a short team name will seeairflow teams syncstart exiting non-zero where it previously succeeded. Defensible, but better as a deliberate call than a side effect.Separately —
re.matchwith a$-anchored pattern accepts a trailing newline, and unliketeams createthis path has no.strip(). See the note on_TEAM_NAMEinenvironment_variables.py.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Answering the question directly, since you asked for it to be a deliberate call rather than a side effect: hard error, and no backport.
This PR is now a draft on 3.4.0 with
backport-to-v3-3-testremoved, and the work moved to #70886. So the "3.3.x patch upgrade starts exiting non-zero" case you flagged does not arise — nothing here ships in a patch release. Given that, the warning option loses its only real justification, and a warning would leave the deployment believing its teams are isolated when they are not.Both halves are implemented in #70886:
teams syncvalidates the stored names alongside the bundle-config ones (Team.get_all_team_namesas you pointed out, one line down and nearly free), and it exits naming the offending names.Your other two points are in there as well — one unanchored pattern used with
re.fullmatchat both sites, defined once in the secrets backend and imported by the CLI, withtest_team_name_pattern_is_shared_with_the_secrets_backendso drift fails CI rather than silently reopening the hole. You were right aboutre.matchand the trailing newline; my first attempt at that test assertedteams createrejects"team1\n"and it does not, because_extract_team_namestrips first. The exposure is on the sync path exactly as you said, and the test moved there.And the finding at the top of this review — the bare-id spelling — I reproduced it. A caller in
team_aasking forprod___dbconnreturnsteam_a___prod's secret. It is the reason this PR is held rather than fixed up: narrowing the team agnostic refusal cannot reach a read that the scoped lookup satisfies. #70886 closes it at the source by makingteam_a___produnspellable, relaxed from forbidding every underscore to forbidding consecutive ones soteam_astays valid.Worth being explicit about one consequence: this PR's own new test asserted
_ab___xmust resolve, and with an unvalidated stored team namedabthat assertion pins the vulnerable behaviour. That is what your comment caught.