New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use SDK harness container for FnAPI jobs when worker_harness_containe… #3434
Conversation
…r_image is not specified. By default the image tag corresponds to the version of the release, same as for legacy images.
R: @aaltay |
get_required_container_version is the right place to update. |
Can you please update dependency.py as Ahmet suggested? Thanks! |
…r_image is not specified. By default the image tag corresponds to the version of the release, same as for legacy images.
0186c7b
to
95daeb4
Compare
Thanks, I moved the logic to dependency.py. I think get_required_container_version is currently meant to return just the version(tag) of the image, so I left it as is and added another function. |
@tvalentyn There is a lint error. Also could you please rebase your change? |
853329a
to
3ffab61
Compare
…r_image is not specified. By default the image tag corresponds to the version of the release, same as for legacy images.
@aaltay Thanks for heads-up. PTAL. |
Tests are failing. Could you look at that? |
ed919e7
to
4232943
Compare
Thanks, my bad. PTAL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also please squash the messy commit tree here so that it is one commit on top of master?
"""For internal use only; no backwards-compatibility guarantees. | ||
|
||
Returns the Google Cloud Dataflow container version for remote execution. | ||
Args: | ||
job_type: string, BEAM job type. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please follow this style guide for structured pydocs when specifying argument / return types: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html (see the other possible style guides here: https://stackoverflow.com/questions/34331088/how-to-comment-parameters-for-pydoc)
Returns: | ||
string, Google Cloud Dataflow container image for remote execution. | ||
""" | ||
if job_type == 'FNAPI_BATCH' or job_type == 'FNAPI_STREAMING': |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add TODO to refactor job types into an enum-like class.
Let's drop this in favor of #3468 |
…r_image is not specified. By default the image tag corresponds to the version of the release, same as for legacy images.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
[BEAM-<Jira issue #>] Description of pull request
mvn clean verify
.<Jira issue #>
in the title with the actual Jira issuenumber, if there is one.
Individual Contributor License Agreement.