Fix RESOURCE_ASSET compatibility with Airflow 2.x in common-compat#64933
Fix RESOURCE_ASSET compatibility with Airflow 2.x in common-compat#64933jedcunningham wants to merge 3 commits intoapache:mainfrom
Conversation
PR apache#63335 hardcoded RESOURCE_ASSET = "Assets" in this compat module, which broke Airflow 2.x deployments. In Airflow 2.x, the equivalent resource is named "Datasets" (RESOURCE_DATASET), not "Assets". apache-airflow-providers-fab imports RESOURCE_ASSET from this module at runtime (via the `else` branch of an `if TYPE_CHECKING` block). When RESOURCE_ASSET resolves to "Assets" instead of "Datasets", it creates duplicate "Assets" and "Datasets" resource types for upgraded instances. And worse "Assets", which dont even exist in AF2, are used for auth checks. This also breaks any custom roles. Fix: use AIRFLOW_V_3_0_PLUS to return the correct value for each version, falling back to RESOURCE_DATASET from airflow.security.permissions (which is not deprecated in Airflow 2.x) for the Airflow 2.x case. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
I guess that means rc2 for common.compat and the two providers that depend on it's new version from the current vote. Thanks for finding and fixing! |
|
BTW. I do think that the Also we will have to find out why this was not detected by compatibility tests - it should have been, but possibly this kind of issue was masked by something. To be diagnosed. |
|
I suspect it wasn't found in the compatibility tests because it caused "Assets" resources and permissions to be created alongside the "Dataset" ones that should have been used. And if you didn't have a custom role (or maybe modified the existing roles) without the dataset perms, its hidden because it "just works", even if its fundamentally wrong. |
Suppress attr-defined (RESOURCE_DATASET doesn't exist in Airflow 3 stubs, where mypy runs) and no-redef (RESOURCE_ASSET defined in both if/else branches) on the Airflow 2.x fallback import. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
There is a mypy issue for providers :( |
There was a problem hiding this comment.
Pull request overview
Fixes a compatibility regression in the common-compat provider where RESOURCE_ASSET was hardcoded to the Airflow 3 resource name, which breaks permission/role behavior on Airflow 2.x.
Changes:
- Introduces
AIRFLOW_V_3_0_PLUSgating to setRESOURCE_ASSETto"Assets"on Airflow 3. - For Airflow 2.x, maps
RESOURCE_ASSETback toairflow.security.permissions.RESOURCE_DATASETto preserve the"Datasets"permission resource name.
| if AIRFLOW_V_3_0_PLUS: | ||
| RESOURCE_ASSET = "Assets" | ||
| else: | ||
| from airflow.security.permissions import ( | ||
| RESOURCE_DATASET as RESOURCE_ASSET, # noqa: F401 # type: ignore[attr-defined, no-redef] | ||
| ) |
There was a problem hiding this comment.
The conditional import of RESOURCE_DATASET happens after module-level assignments (RESOURCE_BACKFILL, etc.), which will trigger Ruff E402 (imports not at top of file) for this module. To avoid CI/lint failures, move the if AIRFLOW_V_3_0_PLUS block (and the import) above the resource constant assignments, or otherwise ensure the import occurs before any non-import statements (alternatively, use a literal string for the AF2 value if you want to avoid conditional importing).
| if AIRFLOW_V_3_0_PLUS: | ||
| RESOURCE_ASSET = "Assets" | ||
| else: | ||
| from airflow.security.permissions import ( | ||
| RESOURCE_DATASET as RESOURCE_ASSET, # noqa: F401 # type: ignore[attr-defined, no-redef] | ||
| ) |
There was a problem hiding this comment.
This change fixes a version-dependent permission constant, but there’s no regression test that asserts the value of RESOURCE_ASSET for the running Airflow major version (the existing import-only test won’t catch the Airflow 2.x vs 3.x mismatch). Add an assertion-based unit test that checks the expected value under Airflow 2.x vs 3.x (e.g., branch on AIRFLOW_V_3_0_PLUS).

Summary
Fixes a regression introduced by #63335, which hardcoded
RESOURCE_ASSET = "Assets"in
providers/common/compat/security/permissions.py. This value is correct forAirflow 3, but breaks Airflow 2.x in two ways:
Wrong resource name in Airflow 2.x.
apache-airflow-providers-fabimportsRESOURCE_ASSETfrom this module at runtime. FAB's role sync updates stock rolesto grant the
"Assets"resource, but the correct Airflow 2.x name is"Datasets".This creates an inconsistency and pollutes the permission model with a resource that
doesn't belong in Airflow 2.x.
Custom roles break. Any role with
"Datasets"permissions stops working, sinceauth checks now look for
"Assets". While you could re-grant permissions against"Assets"after upgrading, existing roles are silently broken.Fix
Gate on
AIRFLOW_V_3_0_PLUS: return"Assets"for Airflow 3, and fall back toRESOURCE_DATASETfromairflow.security.permissions(not deprecated in Airflow 2.x)for Airflow 2.x.
Related
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code