-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Add uv sync --no-dev before provider YAML checks #60728
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add uv sync --no-dev before provider YAML checks #60728
Conversation
jscheffl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks OK for me.
Just a bit unsure - so would request another pair of eyes - I saw it also today (before this PR) that my venv is "wiped" after running soe prek commands... alwas need to re-sync my UV venv... is it intended by pre-commit checks to change / alter venvs? Or shall a temporary secondary be created for this check then?
Otherwise LGTM
bugraoz93
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @Fury0508 , we have recently added an AI Agent section in PRs and saw this PR is 16hours old means while you created it should be there. Could you please fill out that part whether it is used or how degree?
- Add sync_dependencies_without_dev() to strip dev dependencies - Move jsonpath_ng import inside function to avoid ImportError - Helps detect unhandled optional cross-provider dependencies Related: apache#60662
- Exit immediately if uv sync --no-dev fails instead of continuing with warning - Add stdout display for better visibility of sync operation - Ensures validation never runs with dev dependencies present
fbbef26 to
292796f
Compare
|
@bugraoz93 Thanks for pointing that out! I've updated the PR description with the Gen-AI disclosure section. |
Good point! I didn't think about how this would affect local development - only focused on catching the dependency issues in CI. So yeah, wiping the venv every time someone runs pre-commit hooks is definitely annoying. Should this maybe only run in CI and not locally? Or would creating a temp venv for just this check work better? Not sure what's the best fix here - happy to adjust based on what you think makes sense! |
Thanks @Fury0508! |
What change does this PR introduce?
Strips dev dependencies before running provider YAML validation checks by running
uv sync --no-dev --all-packages.Why is this change needed?
Currently, the provider YAML check runs with all dependencies installed, including dev dependencies. This can mask issues where provider code has optional cross-provider dependencies that aren't handled properly (missing try/except blocks for optional imports). By stripping dev dependencies first, we create an environment closer to production and can detect these issues during CI.
Related issue(s)
issue #60662
Changes
sync_dependencies_without_dev()function that runsuv sync --no-dev --all-packagesbefore provider checksTesting
uv sync --no-dev --all-packagesin breeze container to strip dev dependenciesjsonpath-ngfrom amazon provider) remain installedpython scripts/in_container/run_provider_yaml_files_check.pyGen-AI Assisted Contribution
Claude AI was consulted for guidance on Airflow's codebase structure and assistance with resolving CI failures. Core implementation and testing were done independently.