Suspend Apache Beam Provider due to grpcio limitation#61926
Suspend Apache Beam Provider due to grpcio limitation#61926gopidesupavan wants to merge 4 commits intoapache:mainfrom
Conversation
|
af4f6ef to
6e70de6
Compare
a196dae to
d772897
Compare
d772897 to
b348d38
Compare
jscheffl
left a comment
There was a problem hiding this comment.
Hope to resolve this soonish, such that beam can be re-enabled.
eladkal
left a comment
There was a problem hiding this comment.
LGTM but placing temporary block to avoid accidental merge.
it requires also to follow discussion and vote/lazy consensus on mailing list
Yes @eladkal , i am planning to follow that , am trying to resolve some build issues. me and @potiuk had quite a long discussion here https://apache-airflow.slack.com/archives/C015SLQF059/p1770972824043689 |
We can't rely on Slack discussions. |
Yeah correct, sorry for the confusion, am saying will send a email discussion soon, am trying to see after removing beam provider what all impacting. |
Nothing is impacted :) |
Suspending apache-airflow-providers-apache-beam package.
The current Apache Beam Python SDK pins grpcio restricts it to a range like <1.66.
This old grpcio version conflicts with several other dependencies in the project workspace, especially when using uv sync.
Newer versions of opentelemetry-exporter-otlp / opentelemetry-proto that need updated protobuf / grpcio
pydantic-ai-slim (used in common-ai provider) requiring opentelemetry-api ≥ 1.28.0
This blocks development work (including AIP-99) and prevents bumping other dependencies across the repo. the PR is here #61794 where its failing currently due to the limitation of several dependencies pinned.
The grpcio upper bound issue in Apache Beam has been known for a while, and its raised already @potiuk apache/beam#34081 long ago.
The root cause is being addressed upstream in the gRPC project grpc/grpc#37710
(they have already fixed it, with a possible release in grpcio 1.78.1 or a patch that should lift most of these pins/conflicts)
Following the official guidelines for provider suspension:
https://github.com/apache/airflow/blob/main/PROVIDERS.rst#suspending-releases-for-providers
The Apache beam provider can be removed from suspension by a PR reverting this change once grpcio limitation removed from the apache-beam.
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.