Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backend][Plugin] Add support for submitting Custom TrainingJob (framework mode and bring-you-own-container mode) to SageMaker from within Flyte #431

Closed
2 of 6 tasks
bnsblue opened this issue Jul 23, 2020 · 1 comment
Closed
2 of 6 tasks

[Backend][Plugin] Add support for submitting Custom TrainingJob (framework mode and bring-you-own-container mode) to SageMaker from within Flyte #431

bnsblue opened this issue Jul 23, 2020 · 1 comment
Labels
enhancement New feature or request propeller Issues related to flyte propeller
Milestone
0.11.0

Comments

@bnsblue
Copy link
Contributor

bnsblue commented Jul 23, 2020

Why would this plugin be helpful to the Flyte community
This enhancement of the plugin aims to allow the users to leverage the SageMaker Training functionality more comprehensively by enabling them to write custom code or even submit custom-built container to SageMaker.

Type of Plugin

  • Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...)
  • Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...)
  • Customized Plugin using native kubernetes constructs
  • Other

Can you help us with the implementation?

  • Yes
  • No

Additional context
...
@bnsblue bnsblue added the untriaged This issues has not yet been looked at by the Maintainers label Jul 23, 2020
@bnsblue bnsblue mentioned this issue Jul 23, 2020
Merged
8 tasks
@bnsblue bnsblue added enhancement New feature or request propeller Issues related to flyte propeller and removed untriaged This issues has not yet been looked at by the Maintainers labels Jul 24, 2020
This was referenced Jul 24, 2020
Merged
Merged
@kumare3
Copy link
Contributor

kumare3 commented Feb 16, 2021

This is available with Flyte already, marking it to be resovled for 0.11.0

@kumare3 kumare3 added this to the 0.11.0 milestone Feb 16, 2021
@kumare3 kumare3 closed this as completed Feb 16, 2021
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 6, 2022
@pmahindrakar-oss
Store job entry ID for fixed rate scheduler and also fixes cron sched…
32161db 
…ule for missing key (flyteorg#431)

* Store job entry ID for fixed rate scheduler

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

* Add kickoff time input arg only if the user set it in the workflow

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

* Added unit test for no kick off time arg

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
@SandraGH5
Update task_cache.py (flyteorg#431)
f65d318
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Aug 21, 2023
@pmahindrakar-oss
Store job entry ID for fixed rate scheduler and also fixes cron sched…
4080d3f 
…ule for missing key (flyteorg#431)

* Store job entry ID for fixed rate scheduler

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

* Add kickoff time input arg only if the user set it in the workflow

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

* Added unit test for no kick off time arg

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>
eapolinario pushed a commit that referenced this issue Sep 8, 2023
@katrogan
Correct comment for array job max parallelism (#431)
fb13157 
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
eapolinario pushed a commit that referenced this issue Sep 13, 2023
@katrogan @eapolinario
Correct comment for array job max parallelism (#431)
224c594 
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
eapolinario pushed a commit that referenced this issue Sep 26, 2023
@wild-endeavor @honnix
@pingsutw @katrogan @jeevb @hamersaw
GPU Type (#419)
7bd98a9 
* add field

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Pass task execution metadata from agent (#422)

* Pass task execution metadata from agent

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>

* Add doc

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>

* Update protos/flyteidl/admin/agent.proto

Co-authored-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>

* Regenerate

---------

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>
Co-authored-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Add tags to execution spec (#414)

* add tags to execution spec

Signed-off-by: Kevin Su <pingsutw@apache.org>

* add tags to execution spec

Signed-off-by: Kevin Su <pingsutw@apache.org>

* add comment

Signed-off-by: Kevin Su <pingsutw@apache.org>

---------

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Correct comment for array job max parallelism (#431)

Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Add the scalar to the operand (#427)

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add selector

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* move selectors from container to task metadata

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* drop only_preferred

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Updating boilerplate to lock golangci-lint version (#435)

Signed-off-by: Daniel Rammer <daniel@union.ai>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add unpartitioned selector

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* refactor

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* refactor

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* fix oneof names

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add build.os for read the docs

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

---------

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>
Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Daniel Rammer <daniel@union.ai>
Co-authored-by: Honnix <honnix@users.noreply.github.com>
Co-authored-by: Kevin Su <pingsutw@gmail.com>
Co-authored-by: Kevin Su <pingsutw@apache.org>
Co-authored-by: Katrina Rogan <katroganGH@gmail.com>
Co-authored-by: Jeev B <jeevb@users.noreply.github.com>
Co-authored-by: Dan Rammer <daniel@union.ai>
pvditt pushed a commit that referenced this issue Dec 29, 2023
@katrogan
Correct comment for array job max parallelism (#431)
c1417ee 
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
pvditt pushed a commit that referenced this issue Dec 29, 2023
@wild-endeavor @honnix
@pingsutw @katrogan @jeevb @hamersaw
GPU Type (#419)
b92acad 
* add field

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Pass task execution metadata from agent (#422)

* Pass task execution metadata from agent

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>

* Add doc

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>

* Update protos/flyteidl/admin/agent.proto

Co-authored-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>

* Regenerate

---------

Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>
Co-authored-by: Kevin Su <pingsutw@gmail.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Add tags to execution spec (#414)

* add tags to execution spec

Signed-off-by: Kevin Su <pingsutw@apache.org>

* add tags to execution spec

Signed-off-by: Kevin Su <pingsutw@apache.org>

* add comment

Signed-off-by: Kevin Su <pingsutw@apache.org>

---------

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Correct comment for array job max parallelism (#431)

Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Add the scalar to the operand (#427)

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add selector

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* move selectors from container to task metadata

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* drop only_preferred

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Updating boilerplate to lock golangci-lint version (#435)

Signed-off-by: Daniel Rammer <daniel@union.ai>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add unpartitioned selector

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* refactor

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* refactor

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* fix oneof names

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add build.os for read the docs

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

---------

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Jeev B <jeevb@users.noreply.github.com>
Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com>
Signed-off-by: Honnix <honnix@users.noreply.github.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Daniel Rammer <daniel@union.ai>
Co-authored-by: Honnix <honnix@users.noreply.github.com>
Co-authored-by: Kevin Su <pingsutw@gmail.com>
Co-authored-by: Kevin Su <pingsutw@apache.org>
Co-authored-by: Katrina Rogan <katroganGH@gmail.com>
Co-authored-by: Jeev B <jeevb@users.noreply.github.com>
Co-authored-by: Dan Rammer <daniel@union.ai>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Apr 30, 2024
@Future-Outlier
force flag (flyteorg#431)
1152c5f 
Signed-off-by: Future Outlier <eric901201@gmai.com>
Co-authored-by: Future Outlier <eric901201@gmai.com>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Apr 30, 2024
@Future-Outlier
force flag (flyteorg#431)
7f80f02 
Signed-off-by: Future Outlier <eric901201@gmai.com>
Co-authored-by: Future Outlier <eric901201@gmai.com>
austin362667 pushed a commit to austin362667/flyte that referenced this issue May 7, 2024
@Future-Outlier @austin362667
force flag (flyteorg#431)
69cf3c2 
Signed-off-by: Future Outlier <eric901201@gmai.com>
Co-authored-by: Future Outlier <eric901201@gmai.com>
robert-ulbrich-mercedes-benz pushed a commit to robert-ulbrich-mercedes-benz/flyte that referenced this issue Jul 2, 2024
@Future-Outlier @robert-ulbrich-mercedes-benz
force flag (flyteorg#431)
f19a880 
Signed-off-by: Future Outlier <eric901201@gmai.com>
Co-authored-by: Future Outlier <eric901201@gmai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request propeller Issues related to flyte propeller
Projects
None yet
Milestone
0.11.0
Development

No branches or pull requests
2 participants
@bnsblue @kumare3