Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature]FlytePropeller should limit the max number of node traversed / executing in one round #476

Closed
1 of 13 tasks
kumare3 opened this issue Aug 18, 2020 · 6 comments
Closed
1 of 13 tasks

[Feature]FlytePropeller should limit the max number of node traversed / executing in one round #476

kumare3 opened this issue Aug 18, 2020 · 6 comments
Assignees
@kumare3
Labels
enhancement New feature or request
Milestone
0.12.0

Comments

@kumare3
Copy link
Contributor

kumare3 commented Aug 18, 2020

Motivation: Why do you think this is important?
It is possible with dynamic nodes, that users could yield thousands of nodes. The upper limit that is enforced is 4k .FlytePropeller wonderously works with these high numbers, but not without impact on latency. Propeller should limit the number of nodes that can be traversed and be concurrently running.

Goal: What should the final outcome look like, ideally?
When users run large graphs, barring size of the status, flyte propeller should have predictable round latency.

Describe alternatives you've considered
Array Jobs. But, array jobs are not supported for all types.

Flyte component

  • Overall
  • Flyte Setup and Installation scripts
  • Flyte Documentation
  • Flyte communication (slack/email etc)
  • FlytePropeller
  • FlyteIDL (Flyte specification language)
  • Flytekit (Python SDK)
  • FlyteAdmin (Control Plane service)
  • FlytePlugins
  • DataCatalog
  • FlyteStdlib (common libraries)
  • FlyteConsole (UI)
  • Other

[Optional] Propose: Link/Inline
NA
@kumare3 kumare3 added enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers labels Aug 18, 2020
@kumare3 kumare3 added this to the 0.9.0 milestone Aug 25, 2020
@datability-io
Copy link

+1

@kumare3 kumare3 modified the milestones: 0.9.0, 0.10.0 Sep 30, 2020
@EngHabu EngHabu assigned schottra and unassigned schottra Nov 4, 2020
@EngHabu EngHabu modified the milestones: 0.10.0, 0.11.0 Nov 4, 2020
@cwormsl2-zz
Copy link

+1

1 similar comment
@ganeshplyft
Copy link

+1

@EngHabu EngHabu removed this from the 0.11.0 milestone Jan 11, 2021
@kumare3 kumare3 self-assigned this Feb 16, 2021
@kumare3 kumare3 added this to the 0.11.0 milestone Feb 16, 2021
@kumare3
Copy link
Contributor Author

kumare3 commented Feb 16, 2021 •
edited
Loading

I have a PR for this in progress and the tests are really promising. I am hoping to close it by End of February

@kumare3 kumare3 removed the untriaged This issues has not yet been looked at by the Maintainers label Feb 16, 2021
@kumare3 kumare3 mentioned this issue Feb 16, 2021
Closed
8 tasks
@kumare3
Copy link
Contributor Author

kumare3 commented Feb 28, 2021

This has been implemented but is blocked on #742. I will move it to 0.12.0

@kumare3 kumare3 modified the milestones: 0.11.0, 0.12.0 Feb 28, 2021
This was referenced Mar 11, 2021
Closed
Merged
@kumare3 kumare3 mentioned this issue Mar 24, 2021
Closed
@kumare3
Copy link
Contributor Author

kumare3 commented Mar 24, 2021

This core functionality is merged, But it is not yet available to all users. This will be done in #862

@kumare3 kumare3 closed this as completed Mar 24, 2021
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 6, 2022
@EngHabu
Use fnv64 to create a hash for child workflow executions (flyteorg#476)
9cf5368 
* Prefix sub-lp exec id with the parent exec-id

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* cleanup

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* cleanup

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Use a CRD-level version instead

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* cleanup

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Use fnv64 to create a hash for child workflow executions

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Move DefinitionVersion to Status of the CRD

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Update to the released flyteplugins

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Regenerate

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* fix unit tests

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* more fixes

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* fix old unit test

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
@SandraGH5
Update custom_task_plugin.py (flyteorg#476)
e452bad
eapolinario added a commit to eapolinario/flyte that referenced this issue Jul 24, 2023
@pmahindrakar-oss @eapolinario
Added start time for supporting restarts for fixed rate schedules (fl…
507ad44 
…yteorg#476)

* Added race skip check

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

* lint

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

* Fixed unit tests

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* Moved to integration test

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* refactored integration test

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* nit : rename to lastTime

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* nit : revert

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* lastTime -> lastExecTime

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* integration test tag

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

---------

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>
Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Aug 9, 2023
@EngHabu
Use fnv64 to create a hash for child workflow executions (flyteorg#476)
60a71e3 
* Prefix sub-lp exec id with the parent exec-id

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* cleanup

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* cleanup

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Use a CRD-level version instead

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* cleanup

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Use fnv64 to create a hash for child workflow executions

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Move DefinitionVersion to Status of the CRD

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Update to the released flyteplugins

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* Regenerate

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* fix unit tests

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* more fixes

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

* fix old unit test

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
eapolinario added a commit to eapolinario/flyte that referenced this issue Aug 21, 2023
@pmahindrakar-oss @eapolinario
Added start time for supporting restarts for fixed rate schedules (fl…
717bcc3 
…yteorg#476)

* Added race skip check

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

* lint

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>

* Fixed unit tests

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* Moved to integration test

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* refactored integration test

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* nit : rename to lastTime

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* nit : revert

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* lastTime -> lastExecTime

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

* integration test tag

Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>

---------

Signed-off-by: Prafulla Mahindrakar <prafulla.mahindrakar@gmail.com>
Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kumare3 kumare3

Labels
enhancement New feature or request
Projects
None yet
Milestone
0.12.0
Development

No branches or pull requests
6 participants
@EngHabu @schottra @cwormsl2-zz @kumare3 @datability-io @ganeshplyft