Create ks prototype for tf-batch-predict #1351
Conversation
|
/assign jlewi |
|
/assign lluunn |
|
I think we'd just like to have a simple prototype for the k8s/job to run the Dataflow job. We want to put all the logic in the jsonnet file so that when the user runs ks generate they get the full spec and can easily customize it. |
There was a problem hiding this comment.
Done. PTAL
Reviewable status: 0 of 1 files reviewed, all discussions resolved (waiting on @kunmingg and @richardsliu)
| }, | ||
| }, // tfBatchPredictContainerBase | ||
|
|
||
| tfBatchPredictContainer+: base.parts.tfBatchPredictContainerBase + |
There was a problem hiding this comment.
Lets simplify the prototype further. There's not much reason to pull out tfBatchPredictContainer and then override by merging in a dictionary; just paste the relevant dictionary describing the container into the container arrays below line 111.
We want the examples to be optimized for readability. The idea is that if people want to modify them (e.g. they want to set appropriate keys for AWS they would delete and replace the appropriate lines.
In other words, think of this as an "example" that people will copy and modify.
Ideally the example should run under some suitable set of conditions; e.g. assuming there is a valid GCP credential either as a VM service account or as a secret key if secretName is set.
I think we should keep conditionals for setting the following based on parameters
- num GPUs
- GCP secret.
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jlewi The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
* adding tf-batch-predict prototype * first version of ks prototype * change Readme * more clean up * adding an tf-batch-prediction jsonnet file into example * an almost working version * working version * remove the code in kubeflow/kubeflow/tf-batch-predict * simplify the jsonnet by removing unneeded nested structure.
This change is