[BEAM-1218] Removing some of the dataflow references.#1774
[BEAM-1218] Removing some of the dataflow references.#1774aaltay wants to merge 2 commits intoapache:python-sdkfrom
Conversation
|
R: @davorbonaci Please review. |
|
Refer to this link for build results (access rights to CI server needed): Failed Tests: 1beam_PreCommit_Java_MavenInstall/org.apache.beam:beam-runners-apex: 1--none-- |
|
Refer to this link for build results (access rights to CI server needed): |
davorbonaci
left a comment
davorbonaci
left a comment
There was a problem hiding this comment.
Looks good; left a few comments.
| # limitations under the License. | ||
| # | ||
|
|
||
| """Dataflow credentials and authentication.""" |
There was a problem hiding this comment.
All, or at least parts of this, might be Dataflow / GCP specific.
There was a problem hiding this comment.
Correct. Reverted this for now. I will tackle along with the future cleanups.
| Map transform will get on each call *one* row of the main table and *all* rows | ||
| of the side table. The execution framework may use some caching techniques to | ||
| share the side inputs between calls in order to avoid excessive reading:: | ||
| (common case) is expected to be massive and the execution framework will make |
There was a problem hiding this comment.
you are talking about what execution engine will do?
if you are using native source, it is runner-specific.
if not, then it is the source guaranteeing that.
There was a problem hiding this comment.
Rewrote this part to be more generic, and easier for the user to understand.
|
Thank you @davorbonaci please take another look. |
|
Refer to this link for build results (access rights to CI server needed): |
|
LGTM. Merged. |
|
Thank you. |
3 participants
No description provided.