Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a rate limiter to Hive Client #213

Open
1 of 13 tasks
EngHabu opened this issue Mar 17, 2020 · 3 comments
Open
1 of 13 tasks

Add a rate limiter to Hive Client #213

EngHabu opened this issue Mar 17, 2020 · 3 comments
Labels
enhancement New feature or request

Comments

@EngHabu
Copy link
Contributor

EngHabu commented Mar 17, 2020

Motivation: Why do you think this is important?
Implement this TODO: https://github.com/lyft/flyteplugins/blob/master/go/tasks/plugins/hive/client/qubole_client.go#L70

Goal: What should the final outcome look like, ideally?
A configurable rate limiter for qubole/hive client

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Flyte component

  • Overall
  • Flyte Setup and Installation scripts
  • Flyte Documentation
  • Flyte communication (slack/email etc)
  • FlytePropeller
  • FlyteIDL (Flyte specification language)
  • Flytekit (Python SDK)
  • FlyteAdmin (Control Plane service)
  • FlytePlugins
  • DataCatalog
  • FlyteStdlib (common libraries)
  • FlyteConsole (UI)
  • Other

[Optional] Propose: Link/Inline
If you have ideas about the implementation please propose the change. If inline keep it short, if larger then you link to an external document.

Additional context
Add any other context or screenshots about the feature request here.

Is this a blocker for you to adopt Flyte
Please let us know if this makes it impossible to adopt Flyte
@EngHabu EngHabu added enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers labels Mar 17, 2020
@kumare3
Copy link
Contributor

kumare3 commented Mar 30, 2020

@EngHabu I think we should add all of this to a ServiceAPI, that way this can be easily extended to all clients

@kumare3 kumare3 removed the untriaged This issues has not yet been looked at by the Maintainers label Oct 22, 2021
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
@hamersaw
Cache Serialize API (flyteorg#213)
801a658 
* updating catalog client to contain reservation handling API

Signed-off-by: Daniel Rammer <hamersaw@protonmail.com>

* added PhaseWaitingForCache to indicate cache reservation exists

Signed-off-by: Daniel Rammer <hamersaw@protonmail.com>

* added ReservationEntry to interface with cache reservation API

Signed-off-by: Daniel Rammer <hamersaw@protonmail.com>

* added support for requesting a heartbeat_interval during GetOrExtendReservation requests to the datacatalog

Signed-off-by: Daniel Rammer <daniel@union.ai>

* fixed lint issues

Signed-off-by: Daniel Rammer <daniel@union.ai>

* replaced flyteidl repo with cache reservation version - need to change back before merge

Signed-off-by: Daniel Rammer <daniel@union.ai>

* upated flyteidl version

Signed-off-by: Daniel Rammer <daniel@union.ai>

* updated flyteidl version

Signed-off-by: Daniel Rammer <daniel@union.ai>

* remove flyteidl replace in go.mod and updating to latest version

Signed-off-by: Daniel Rammer <daniel@union.ai>

* added unit testes to catalog client

Signed-off-by: Daniel Rammer <daniel@union.ai>

* added docs for exports

Signed-off-by: Daniel Rammer <daniel@union.ai>

* fixed lint issues

Signed-off-by: Daniel Rammer <daniel@union.ai>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
@SandraGH5 @samhita-alla
Update fast_registration.py (flyteorg#213)
28223a7 
* Update fast_registration.py

* update fast registration (flyteorg#216)

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* Modify TOC and add a mechanism to consider RST files  (flyteorg#218)

* update fast registration

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* modified TOC and added a mechanism to consider RST files

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* updated conf.py

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* modified TOC

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* edit fast registration (flyteorg#219)

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* Edit fast_registration.rst (flyteorg#220)

* edit fast registration

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* removed unnecessary RST file

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* Fix conflicts in flyteorg#213 (flyteorg#230)

Signed-off-by: Samhita Alla <aallasamhita@gmail.com>

* Revert "Fix conflicts in flyteorg#213 (flyteorg#230)" (flyteorg#231)

This reverts commit 525fbbcfb8828ac2e0b6d75d637455bcff1b9d21.

Co-authored-by: Samhita Alla <aallasamhita@gmail.com>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
@flyte-bot @samhita-alla
Update documentation (flyteorg#213)
c3461e9 
Signed-off-by: Flyte-Bot <admin@flyte.org>

Co-authored-by: samhita-alla <samhita-alla@users.noreply.github.com>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Aug 21, 2023
@hamersaw
Cache Serialize API (flyteorg#213)
360cc23 
* updating catalog client to contain reservation handling API

Signed-off-by: Daniel Rammer <hamersaw@protonmail.com>

* added PhaseWaitingForCache to indicate cache reservation exists

Signed-off-by: Daniel Rammer <hamersaw@protonmail.com>

* added ReservationEntry to interface with cache reservation API

Signed-off-by: Daniel Rammer <hamersaw@protonmail.com>

* added support for requesting a heartbeat_interval during GetOrExtendReservation requests to the datacatalog

Signed-off-by: Daniel Rammer <daniel@union.ai>

* fixed lint issues

Signed-off-by: Daniel Rammer <daniel@union.ai>

* replaced flyteidl repo with cache reservation version - need to change back before merge

Signed-off-by: Daniel Rammer <daniel@union.ai>

* upated flyteidl version

Signed-off-by: Daniel Rammer <daniel@union.ai>

* updated flyteidl version

Signed-off-by: Daniel Rammer <daniel@union.ai>

* remove flyteidl replace in go.mod and updating to latest version

Signed-off-by: Daniel Rammer <daniel@union.ai>

* added unit testes to catalog client

Signed-off-by: Daniel Rammer <daniel@union.ai>

* added docs for exports

Signed-off-by: Daniel Rammer <daniel@union.ai>

* fixed lint issues

Signed-off-by: Daniel Rammer <daniel@union.ai>
@github-actions
Copy link

Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏

@github-actions github-actions bot added the stale label Aug 26, 2023
@github-actions
Copy link

github-actions bot commented Sep 2, 2023

Hello 👋, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 2, 2023
@eapolinario eapolinario reopened this Nov 2, 2023
@github-actions github-actions bot removed the stale label Nov 3, 2023
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Apr 30, 2024
@flyte-bot @samhita-alla
Update documentation (flyteorg#213)
3c0eaa2 
Signed-off-by: Flyte-Bot <admin@flyte.org>

Co-authored-by: samhita-alla <samhita-alla@users.noreply.github.com>
austin362667 pushed a commit to austin362667/flyte that referenced this issue May 7, 2024
@flyte-bot @samhita-alla @austin362667
Update documentation (flyteorg#213)
c08c609 
Signed-off-by: Flyte-Bot <admin@flyte.org>

Co-authored-by: samhita-alla <samhita-alla@users.noreply.github.com>
robert-ulbrich-mercedes-benz pushed a commit to robert-ulbrich-mercedes-benz/flyte that referenced this issue Jul 2, 2024
@flyte-bot @samhita-alla @robert-ulbrich-mercedes-benz
Update documentation (flyteorg#213)
78ac611 
Signed-off-by: Flyte-Bot <admin@flyte.org>

Co-authored-by: samhita-alla <samhita-alla@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@EngHabu @eapolinario @kumare3