Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump apache-airflow-providers-cncf-kubernetes version up to 8.0.0 #37899

Closed

Conversation

moiseenkov
Copy link
Contributor

Bump missing version for apache-airflow-providers-cncf-kubernetes up to 8.0.0 required for the #36847.

related: #36847, #37890

@eladkal
Copy link
Contributor

eladkal commented Mar 5, 2024

We don't normally bump min versions for each feature, that would cause all providers to work only with latest version. I think where required we are raising exception to let use know they need newer version of the depended provider.

@moiseenkov
Copy link
Contributor Author

@eladkal , thank you for fast reply.
Could you please clarify this for me, because I haven't seen these kinds of exceptions before and I'm not aware of this approach? In which cases we should (and shouldn't) bump the provider package's dependencies? Is there any docs?

@eladkal
Copy link
Contributor

eladkal commented Mar 5, 2024

Example:

raise AirflowException(
f"You are trying to use common-sql with {hook.__class__.__name__},"
f" but the Hook class comes from provider {provider} that does not support it."
f" Please upgrade provider {provider} to at least {min_version}."
)

raise AirflowException(
f"You are trying to use `common-sql` with {hook.__class__.__name__},"
" but its provider does not support it. Please upgrade the provider to a version that"
" supports `common-sql`. The hook class should be a subclass of"
" `airflow.providers.common.sql.hooks.sql.DbApiHook`."
f" Got {hook.__class__.__name__} Hook with class hierarchy: {hook.__class__.mro()}"
)

@potiuk
Copy link
Member

potiuk commented Mar 5, 2024

And to add - what @eladkal explained is even better protection and more user-friendly way of adding the version dependency. The problem with the cross-provider dependencies is that they are soft. They are not - and cannot be enforced, so they are mostly informative. The only way you can make such cross-dependency works is when you do

pip install apache-airflow-providers-google[cncf.kubernetes]

When you do:

pip install apache-airflow-providers-google
pip install apache-airflow-providers-cncf-kubernetes

or

pip install apache-airflow[google,cncf.kubernetes]

there is absolutely no way to enforce this extra limit based on what is written in the extra. So checking the version of provider (or failing on importing non-existing value and raising exception is way better and more user-friendly approach..

BTW. We hava a dedicated exception OptionalProviderFeatureException - that can be used for that purpose.

@moiseenkov
Copy link
Contributor Author

Thank you so much for your explanation! Our team will follow these guides.

@moiseenkov moiseenkov closed this Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants