Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unify engine initialization in pipeline #1085

Merged
merged 2 commits into from
Feb 4, 2024
Merged

Conversation

irexyc
Copy link
Collaborator

@irexyc irexyc commented Jan 31, 2024

After PR 1064

Motivation

既然 turbomind 加载 hf 模型的时候已经不依赖 model_name 了,那么 pipeline 初始化 turbomind engine 的时候,对于对话模版的处理,应该同 pytorch engine 一致。

修改之后,对于使用 pipeline 接口而言,pytorch / turbomind 的行为是一致的了,对话模版由 pipeline 处理。

@lvhan028
Copy link
Collaborator

turbomind.py中,是不是可以把对话模板拿掉了呢?

@irexyc
Copy link
Collaborator Author

irexyc commented Feb 1, 2024

turbomind.py中,是不是可以把对话模板拿掉了呢?

会再提一个pr,修改 turbomind.py 和 turbomind.chat

Copy link
Collaborator

@AllentDan AllentDan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这块可以合,#1095 还会改这块的逻辑。想法是能够直接去掉其他所有model_name 的 arg,只保留 chat_template 的model_name。这样最干净,只要考虑一个输入就行。

@lvhan028 lvhan028 merged commit 6844c2b into InternLM:main Feb 4, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants