-
Notifications
You must be signed in to change notification settings - Fork 376
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support user-specified IMAGE_TOKEN position for deepseek-vl model #1627
Conversation
may provide a case so that @zhulinJulia24 can add it into TCs |
logger.error( | ||
f'the number of {IMAGE_TOKEN} is not equal ' | ||
f'to input images, {len(segs) - 1} vs {len(features)}') | ||
features = features[:len(segs) - 1] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible that len(segs) > len(features)
docs/zh_cn/inference/vl_pipeline.md
Outdated
from lmdeploy.vl import load_image | ||
from lmdeploy.vl.constants import IMAGE_TOKEN | ||
|
||
pipe = pipeline('deepseek-ai/deepseek-vl-7b-chat') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shall we try deepseek-ai/deepseek-vl-1.3b-chat
instead as an example? It is smaller and can be downloaded faster.
```python | ||
from lmdeploy import pipeline | ||
from lmdeploy.vl import load_image | ||
from lmdeploy.vl.constants import IMAGE_TOKEN |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May notify users that is is actually '<IMAGE_TOKEN>'
in comments or other forms. This is useful for api_server users.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Motivation
for some models, the location of the image token is not restricted, in this case, the user should insert the image token manually.