[Monorepo] - Import flyteidl #4017
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add new phases to Task execution
* Add an ErrorKind field in ContainerError * Bump versions
* Add FailureHandlingStrategy to workflow idl * Rename and allow overrides per LaunchPlan and Execution * Revert override changes * revert
* Only keep options we will implement at the moment * regenerate
* IDL changes for Node-Node relationship refactor * Ability to provide information of the parent node. * Introduction of group_id to indicate grouping of nodes within the Parent node - (instead of dummy nodes) * Support for querying all nodes under node with group. * Support for node_id to point to original node in the workflow/graph, and node_name
…#70) * Catalog store handling * Event proto updated * generated * merged * Dataset Identifier changed * Better model * Exposing TaskNodeMetadata with catalog information in the API * 0.17.38 * Comment * docs updated
…in algorithms) and HyperparameterTuningJob (#66) * adding sagemaker protos Co-authored-by: Haytham AbuelFutuh <habuelfutuh@lyft.com>
* first cut * bump version * add to project.json * regen Co-authored-by: Konstantin Gizdarski <kgizdarski@lyft.com> Co-authored-by: Katrina Rogan <krogan@lyft.com>
* add project update endpoint * PR comments * pr comments * pr comments (last one ;p) * spelling * generate Co-authored-by: Konstantin Gizdarski <kgizdarski@lyft.com> Co-authored-by: Katrina Rogan <krogan@lyft.com>
This PR fixes Stackdriver log link using `resource.labels.pod_name` as `advancedFilter` instead of `logName`.
* Support cron schedule with offset. For example: https://github.com/spotify/styx#schedule-string * Add generated files * Update protos/flyteidl/admin/schedule.proto Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Update protos/flyteidl/admin/schedule.proto Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Update protos/flyteidl/admin/schedule.proto Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Regenerate files * Regenerate files * Change the message name back * Revert "Change the message name back" This reverts commit e468ea836dd30fc01699fd417ee64a182603b7f7. * Add comments for schedule and offset Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
…ing (#84) * `distributed_protocol` added as a field in training job resource config
* Add backend plugin system service Signed-off-by: Kevin Su <pingsutw@apache.org> * Add backend plugin system service Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * update state Signed-off-by: Kevin Su <pingsutw@apache.org> * update state Signed-off-by: Kevin Su <pingsutw@apache.org> * dics Signed-off-by: Kevin Su <pingsutw@apache.org> * Remove output prefix from get request Signed-off-by: Kevin Su <pingsutw@apache.org> * update Signed-off-by: Kevin Su <pingsutw@apache.org> * remove prev state Signed-off-by: Kevin Su <pingsutw@apache.org> * update proto Signed-off-by: Kevin Su <pingsutw@apache.org> * remove error message Signed-off-by: Kevin Su <pingsutw@apache.org> * update comment Signed-off-by: Kevin Su <pingsutw@apache.org> * make generate Signed-off-by: Kevin Su <pingsutw@apache.org> * Rename the service Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
* Generate rust protobuf using buf Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Commit generated rust code Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
* Move to official remote plugins Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Regenerate python files Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
* Add elastic config args to pytorch proto Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Add elastic config message type for torchrun training Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> --------- Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> Co-authored-by: Fabio Grätz <fabiogratz@googlemail.com> Co-authored-by: Ketan Umare <ketan.umare@gmail.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> #minor
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> #minor
…ecs for different replica groups (#386) * refactor kubeflow operators proto Signed-off-by: Yubo Wang <yubwang@linkedin.com> * add back the original proto for backward compatible Signed-off-by: Yubo Wang <yubwang@linkedin.com> * clean up comments Signed-off-by: Yubo Wang <yubwang@linkedin.com> * add kubeflow.rs Signed-off-by: Yubo Wang <yubwang@linkedin.com> * add elastic config Signed-off-by: Yubo Wang <yubwang@linkedin.com> * add command to MPI Signed-off-by: Yubo Wang <yubwang@linkedin.com> * add slots and command to mpi spec Signed-off-by: Yubo Wang <yubwang@linkedin.com> --------- Signed-off-by: Yubo Wang <yubwang@linkedin.com> Co-authored-by: Yubo Wang <yubwang@linkedin.com>
Signed-off-by: byhsu <byhsu@linkedin.com> Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: byhsu <byhsu@linkedin.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
* add support for Union Scalar Signed-off-by: Yubo Wang <yubwang@linkedin.com> * support union type and literals Signed-off-by: Yubo Wang <yubwang@linkedin.com> * change union type extraction Signed-off-by: Yubo Wang <yubwang@linkedin.com> --------- Signed-off-by: Yubo Wang <yubwang@linkedin.com> Co-authored-by: Yubo Wang <yubwang@linkedin.com> Co-authored-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: byhsu <byhsu@linkedin.com> Co-authored-by: byhsu <byhsu@linkedin.com>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: byhsu <byhsu@linkedin.com>
* Rename externalPluginService to AgentService Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
* Add external-plugin-service proto back to the idl Signed-off-by: Kevin Su <pingsutw@apache.org> * update idl Signed-off-by: Kevin Su <pingsutw@apache.org> * update idll Signed-off-by: Kevin Su <pingsutw@apache.org> * update idll Signed-off-by: Kevin Su <pingsutw@apache.org> * AsyncAgentService Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
* added initial ArrayNode definition Signed-off-by: Daniel Rammer <daniel@union.ai> * updated parallelism and min_successes int types Signed-off-by: Daniel Rammer <daniel@union.ai> * updated min_successes type Signed-off-by: Daniel Rammer <daniel@union.ai> * updated docs Signed-off-by: Daniel Rammer <daniel@union.ai> --------- Signed-off-by: Daniel Rammer <daniel@union.ai> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
* Pass task execution metadata from agent Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> * Add doc Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> * Update protos/flyteidl/admin/agent.proto Co-authored-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Honnix <honnix@users.noreply.github.com> * Regenerate --------- Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> Signed-off-by: Honnix <honnix@users.noreply.github.com> Co-authored-by: Kevin Su <pingsutw@gmail.com>
* add tags to execution spec Signed-off-by: Kevin Su <pingsutw@apache.org> * add tags to execution spec Signed-off-by: Kevin Su <pingsutw@apache.org> * add comment Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Daniel Rammer <daniel@union.ai>
…orepo--import-flyteidl Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
pingsutw
approved these changes
Sep 8, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe your changes
This PR imports flyteidl as a top-level component in the monorepo. Subsequent PRs are going to fix the module name, fix single-binary compilation, etc.
This brings flyteorg/flyteidl@7f62a15.