Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support mini gemini llama #1438

Merged
merged 16 commits into from
Apr 19, 2024
Merged

Support mini gemini llama #1438

merged 16 commits into from
Apr 19, 2024

Conversation

AllentDan
Copy link
Collaborator

No description provided.

@AllentDan AllentDan added the WIP label Apr 16, 2024
@AllentDan AllentDan mentioned this pull request Apr 16, 2024
Closed
@AllentDan AllentDan removed the WIP label Apr 17, 2024
@AllentDan
Merge branch 'main' into mini-gemini
47e0186 
Conflicts:
	docs/en/supported_models/supported_models.md
	docs/zh_cn/supported_models/supported_models.md
@lvhan028 lvhan028 added the enhancement New feature or request label Apr 17, 2024
@lvhan028 lvhan028 self-requested a review April 17, 2024 12:18
@lvhan028 lvhan028 requested a review from irexyc April 19, 2024 06:09
@lvhan028
Copy link
Collaborator

lvhan028 commented Apr 19, 2024 •
edited

  • test pipeline with Mini-Gemini-7B. tp=1,2,4
  • api_server, use openai client
  • gradio
    when I give a customized model name to gradio, it acts unexpectedly
lmdeploy serve gradio /workspace/140_models/mini_gemini/Mini-Gemini-7B/ --server-port 33337 --tp 2 --model-name minigemini-7b

image url: https://raw.githubusercontent.com/QwenLM/Qwen-VL/master/assets/demo.jpeg

@lvhan028
Copy link
Collaborator

UT failed

@lvhan028
Copy link
Collaborator

lvhan028 commented Apr 19, 2024 •
edited

gradio 测试失败。
输入第一张图 + prompt，正常
继续问问题，webui显示一直在processing。后台看，server没在推理。有两行日志信息：

2024-04-19 06:57:20,748 - lmdeploy - INFO - prompt: 明明是东方明珠
2024-04-19 06:57:20,749 - lmdeploy - INFO - ImageEncoder received 0 images, left 0 images.

@lvhan028
Copy link
Collaborator

我的gradio版本是3.50.2

@lvhan028
Copy link
Collaborator

好像 gradio 没支持自定义模型名字，之前只做了 api_server 的，需要也支持吗？

不用。不过为啥没有报错呢？

@AllentDan
Copy link
Collaborator Author

卡在没有图片的输入的情况写了 image_token 进去，修好了

@AllentDan AllentDan mentioned this pull request Apr 19, 2024
Merged
@irexyc
Copy link
Collaborator

irexyc commented Apr 19, 2024

README 里面也有 Supported Models，要加上 MiniGeminiLlama 么

irexyc
irexyc reviewed Apr 19, 2024
View reviewed changes
lmdeploy/vl/model/utils.py Outdated Show resolved Hide resolved
@lvhan028 lvhan028 merged commit 17e937d into InternLM:main Apr 19, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lvhan028 lvhan028 lvhan028 approved these changes

@irexyc irexyc irexyc approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@AllentDan @lvhan028 @irexyc