-
Notifications
You must be signed in to change notification settings - Fork 245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update docker docs for VL api #1534
Conversation
@vody-am If you are familiar with Chinese, could you please update the user guide in |
@lvhan028 Google translate has been working well, so yes I will give it a shot 😂 |
@lvhan028 while I am here -- is there anything special one has to set to use multiple GPUs? Thus far I have experimented with one container per GPU, but it seems like there should be a way to use multiple cards. Is it the |
Yes, by setting |
Hi, @vody-am
|
|
||
COPY . . | ||
|
||
CMD ["lmdeploy", "serve", "api_server", "liuhaotian/llava-v1.6-34b"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In a real production environment, the deployed model path is usually stored on something like Open Storage Service. Almost never download and load from HuggingFace every time you deploy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Therefore, this is more of a simple reference and has little significance in actual production environments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that's a fair assessment.
Docs update in relation to issue #1514
Motivation
Updating documentation around Docker images for new users.
It should be made clear to users what is expected, and a way to resolve issues described.
Modification
Adding a warning about dependencies and an example of how to resolve by creating a new Docker image.