-
Notifications
You must be signed in to change notification settings - Fork 269
LLMPipeline(NPU): Configuration section to fine-tune LM head model #2317
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LLMPipeline(NPU): Configuration section to fine-tune LM head model #2317
Conversation
b768824
to
fec1d2e
Compare
f2a1f7a
to
a5d485e
Compare
src/cpp/src/utils.cpp
Outdated
|
||
rename_key(config, "++PREFILL_CONFIG", "++NPUW_LLM_PREFILL_CONFIG"); | ||
rename_key(config, "++GENERATE_CONFIG", "++NPUW_LLM_GENERATE_CONFIG"); | ||
rename_key(config, "++TAIL_MM_CONFIG", "++NPUW_LLM_SHARED_HEAD_CONFIG"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do they do? I can't find any other mention in this repo and in Google. I guess someone familiar with that would be a better reviewer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got you!
### Details: - *Separated Vocab MatMul into additional model* - *Called infer for additional model in LLMInferRequest* ### Dependencies: - *OpenVINO GenAI PR: openvinotoolkit/openvino.genai#2317 ### Tickets: - *EISW-165218*
This PR needs to be updated now I believe |
### Details: - *Separated Vocab MatMul into additional model* - *Called infer for additional model in LLMInferRequest* ### Dependencies: - *OpenVINO GenAI PR* : openvinotoolkit/openvino.genai#2317 ### Tickets: - *EISW-165218* ### Note: - *Regression was found with the PR in the previous merge, PR was reverted, this regression got fixed in current PR* --------- Co-authored-by: Dmitry Matveev <dmitry.matveev@intel.com>
ee9a7fe
to
a70cdb1
Compare
src/cpp/src/utils.cpp
Outdated
// Temporarily, for GenAI testing in CI, original update will be in OpenVINO: | ||
update_config(config, {"NPUW_LLM_SHARED_HEAD", "YES"}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this particular line be merged here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think so, I will remove it!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
did you?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed, thanks!
5357656
OpenVINO GenAI logic for enablement of Three-model pipeline on NPU: openvinotoolkit/openvino#30554