Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harden Puller logic in the side-car Agent #1048

Closed
ifilonenko opened this issue Aug 27, 2020 · 2 comments · Fixed by #1055 or #1105
Closed

Harden Puller logic in the side-car Agent #1048

ifilonenko opened this issue Aug 27, 2020 · 2 comments · Fixed by #1055 or #1105

Comments

@ifilonenko
Copy link
Contributor

ifilonenko commented Aug 27, 2020

/kind feature

Describe the solution you'd like
With the merging of #989 a variety of enhancements are still required before we can remove the [POC] label from the multi-model puller. These enhancements include but are not limited to:

  • Handle retries
  • Move logger to zapr
  • Set proper buffer size
  • Handle request logic
  • Implement a variety of protocols: HDFS, GCS, FILE, HTTPS
  • Handle proper config overrides of s3 clients
  • Remove need for timestamp for config diffing by using a bool
  • Figure out if mid-download failures can be checkpointed and recovered *optional
  • loop through map in a parallel way instead of sequential *optional
  • Handle load, unload, load events *optional

Anything else you would like to add:

  • Unit tests
  • End to End tests
@issue-label-bot
Copy link

Issue-Label Bot is automatically applying the labels:

Label Probability
area/engprod 0.57

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

@issue-label-bot
Copy link

Issue Label Bot is not confident enough to auto-label this issue.
See dashboard for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants