Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support model_name customization for api_server #1403

Merged
merged 2 commits into from
Apr 16, 2024

Conversation

AllentDan
Copy link
Collaborator

@lvhan028 lvhan028 self-requested a review April 11, 2024 12:24
@lvhan028
Copy link
Collaborator

我感觉这个方案不太好,像 uuid 这种不能提供任何有用信息。

如果 vllm,fastchat等有类似的功能,我们可以参考。
如果没有的话,我的想法是区分 --model-name, --chat-template
--model-name 表示 v1/models 结果返回的结果
--chat-template 表示对话模板的名字

可以有BC

@AllentDan
Copy link
Collaborator Author

AllentDan commented Apr 15, 2024

random_uuid 只用于获取对话模板,用户无感知。现在用户可以传任意字符给 --model-name,然后用于 /v1/chat/completion

@lvhan028
Copy link
Collaborator

了解了。

@lvhan028 lvhan028 requested a review from irexyc April 15, 2024 06:56
model_name=args.model_name,
meta_instruction=args.meta_instruction,
capability=args.cap)
chat_template_config = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meta_instruction, capability 都没用了是么?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

想把他们去掉,之前已经有 deprecate 一段时间了

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

去掉,我觉得也可以。merge下 latest main,看看 pr-test 的测试结果,可能有测试用例要改。

@lvhan028 lvhan028 changed the title Customize model name for api_server Support model_name customization for api_server Apr 16, 2024
@lvhan028 lvhan028 merged commit 4df476f into InternLM:main Apr 16, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants