-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BEAM-13812] Integrate DataprocClusterManager into Interactive Environment #16741
[BEAM-13812] Integrate DataprocClusterManager into Interactive Environment #16741
Conversation
R: @KevinGG |
Codecov Report
@@ Coverage Diff @@
## master #16741 +/- ##
==========================================
+ Coverage 74.65% 83.63% +8.98%
==========================================
Files 656 453 -203
Lines 82424 62470 -19954
==========================================
- Hits 61532 52249 -9283
+ Misses 19892 10221 -9671
+ Partials 1000 0 -1000
Continue to review full report at Codecov.
|
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_beam.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_environment.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_runner.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_runner.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_environment.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_environment.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_beam.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_beam.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_runner.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_runner.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Show resolved
Hide resolved
1eb3a88
to
1fca0d9
Compare
sdks/python/apache_beam/runners/interactive/interactive_runner.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_runner.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_runner.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
863ab52
to
feaac86
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/runners/interactive/interactive_runner.py
Outdated
Show resolved
Hide resolved
7e2077a
to
3ac8a4c
Compare
retest this please |
c385703
to
5dbb9da
Compare
e99ebcb
to
8da4e2c
Compare
retest this please |
7888ae8
to
f0ca9db
Compare
1ae2f5c
to
3cbc708
Compare
Run Python PreCommit |
colab users are reporting error logs showing "No cluster_manager is associated with the provided pipeline!". Could you please file a bug / fix the problem? |
Hi @aaltay, Thanks for letting me know. I am working on a fix in this PR: #16936. Here is the link to the Beam ASF thread: BEAM-13996. |
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username
).[BEAM-XXX] Fixes bug in ApproximateQuantiles
, where you replaceBEAM-XXX
with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.CHANGES.md
with noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
ValidatesRunner
compliance status (on master branch)Examples testing status on various runners
Post-Commit SDK/Transform Integration Tests Status (on master branch)
Pre-Commit Tests Status (on master branch)
See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.