Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always download the model weights when pod starts #88

Closed
kerthcet opened this issue Aug 16, 2024 · 2 comments · Fixed by #102
Closed

Always download the model weights when pod starts #88

kerthcet opened this issue Aug 16, 2024 · 2 comments · Fixed by #102
Labels
bug Categorizes issue or PR as related to a bug. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
Milestone

Comments

@kerthcet
Copy link
Member

What happened:

Because of InftyAI/omnistore#12, we'll always download the model weights even we cached them in the host machine.

What you expected to happen:

When model weights downloaded, we should not reload again.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version):
  • LWS version:
  • llmaz version (use git describe --tags --dirty --always):
  • Cloud provider or hardware configuration:
  • OS (e.g: cat /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:
@kerthcet kerthcet added the bug Categorizes issue or PR as related to a bug. label Aug 16, 2024
@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. labels Aug 16, 2024
@kerthcet
Copy link
Member Author

/milestone v0.1.0

@InftyAI-Agent InftyAI-Agent added this to the v0.1.0 milestone Aug 16, 2024
@kerthcet
Copy link
Member Author

Let's not cache the models to the host right now because it makes little sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Categorizes issue or PR as related to a bug. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants