Fix conn-fields check crash for nested provider packages#70224
Conversation
boring-cyborg
Bot
added
area:dev-tools
backport-to-v3-3-test
50d2287 to
02d5829
Compare
|
@dabla did you request changes by mistake? |
02d5829 to
3a7c18c
Compare
|
yep, all fixed |
Yes sorry for that. |
|
I created a PR which fixes the selective checks issues, so that when a change is done to the static checks of the providers but no providers where changed, it will still run the static checks on the providers. |
|
need another fix related to jira. will update it soon |
3a7c18c to
f5a352b
Compare
|
jira fix is up. let's see how CI goes (even though it won't really run without #70230) |
check_provider_yaml_files.py mapped any changed Python file to "<first-path-segment>/provider.yaml", which breaks for namespace packages (apache, atlassian, common, cncf, dbt, microsoft) that nest an extra directory level, e.g. a change under apache/beam/... resolved to the nonexistent apache/provider.yaml instead of apache/beam/provider.yaml. Reuse the existing KNOWN_SECOND_LEVEL_PATHS list from common_prek_utils to resolve the correct nesting depth.
f5a352b to
b07ed12
Compare
Backport failed to create: v3-3-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker c578603 v3-3-testThis should apply the commit to the v3-3-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
apache#70224 merged a few hours after this wave was cut, declaring a Verify SSL connection field and hidden-field behaviour for the Jira connection. That metadata ships to users through get_provider_info.py, so the provider needs a release for the connection form to gain the field.
apache#70224 merged a few hours after this wave was cut, declaring a Verify SSL connection field and hidden-field behaviour for the Jira connection. That metadata ships to users through get_provider_info.py, so the provider needs a release for the connection form to gain the field.
8 participants
Why
The conn-fields static check crashed for any change under a namespace provider directory —
apache,atlassian,common,cncf,dbt,microsoft. These nest the real provider one level deeper (e.g. the provider lives atapache/beam, notapache), but_resolve_provider_yaml_files()incheck_provider_yaml_files.pyalways took the first path segment as the provider name. A change underapache/beam/...resolved to the nonexistentapache/provider.yaml, sorun_provider_yaml_files_check.pyraisedFileNotFoundErrorin https://github.com/apache/airflow/actions/runs/29899861660/job/88860560132What
The fix reuses the existing
KNOWN_SECOND_LEVEL_PATHSconstant fromcommon_prek_utils— the same list already drivingmypy_folder.pyandcheck_providers_subpackages_all_have_init.py— to pick the correct nesting depth, instead of hand-rolling a second copy of that list that would drift.Was generative AI tooling used to co-author this PR?
Generated-by: [Claude] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.