Fix Google Dataflow hook failing import when apache-beam not installed#65659
Merged
shahar1 merged 3 commits intoapache:mainfrom Apr 22, 2026
Merged
Fix Google Dataflow hook failing import when apache-beam not installed#65659shahar1 merged 3 commits intoapache:mainfrom
shahar1 merged 3 commits intoapache:mainfrom
Conversation
boring-cyborg
Bot
added
area:providers
provider:google
eladkal
reviewed
Apr 22, 2026
eladkal
reviewed
Apr 22, 2026
eladkal
approved these changes
Apr 22, 2026
potiuk
approved these changes
Apr 22, 2026
DataflowHook imported BeamHook, BeamRunnerType, and beam_options_to_args at module level, making the entire google provider fail to import when apache-airflow-providers-apache-beam is not installed. apache-beam is not a declared dependency of the google provider, so this hard import was always incorrect. The import error surfaced during the provider release process (apache#65614) when the provider yaml check script imports all registered provider modules and beam was not present in the environment. Move the imports to lazy form inside the two methods that actually use them: DataflowHook.__init__ and DataflowHook._build_gcloud_command.
shahar1
force-pushed
the
fix/google-dataflow-beam-hard-import
branch
from
1c7648d to
a39983b
Compare
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
59 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DataflowHookimportedBeamHook,BeamRunnerType, andbeam_options_to_argsat module level fromairflow.providers.apache.beam. Sinceapache-airflow-providers-apache-beamis not a declared dependency of the google provider, this hard import causes aModuleNotFoundErrorwhenever the google provider is imported in an environment where beam is not installed.The issue was discovered during the provider release process (#65614), where the
check-provider-yaml-validpre-commit hook imports all modules registered inprovider.yamlto verify class registrations. When beam was absent from the environment, the hook aborted with:The fix moves the two beam imports to lazy form inside the methods that actually require them (
DataflowHook.__init__andDataflowHook._build_gcloud_command), so importing the module has no dependency on beam being installed.Was generative AI tooling used to co-author this PR?
Generated-by: GitHub Copilot (Claude Sonnet 4.6) following the guidelines