Skip to content

LLMPipeline(NPU): Configuration section to fine-tune LM head model #2317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Aug 11, 2025

Conversation

AsyaPronina
Copy link
Contributor

OpenVINO GenAI logic for enablement of Three-model pipeline on NPU: openvinotoolkit/openvino#30554

@dmatveev dmatveev changed the title Added ability to specify config for Tail Vocab Matmul model NPU: Configuration section to fine-tune tail Jun 6, 2025
@dmatveev dmatveev changed the title NPU: Configuration section to fine-tune tail LLMPipeline(NPU): Configuration section to fine-tune tail Jun 6, 2025
@AsyaPronina AsyaPronina force-pushed the three_model_pipeline branch from b768824 to fec1d2e Compare June 25, 2025 14:42
@AsyaPronina AsyaPronina force-pushed the three_model_pipeline branch from f2a1f7a to a5d485e Compare July 15, 2025 00:55
@AsyaPronina AsyaPronina requested a review from Wovchena July 22, 2025 23:08

rename_key(config, "++PREFILL_CONFIG", "++NPUW_LLM_PREFILL_CONFIG");
rename_key(config, "++GENERATE_CONFIG", "++NPUW_LLM_GENERATE_CONFIG");
rename_key(config, "++TAIL_MM_CONFIG", "++NPUW_LLM_SHARED_HEAD_CONFIG");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do they do? I can't find any other mention in this repo and in Google. I guess someone familiar with that would be a better reviewer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got you!

github-merge-queue bot pushed a commit to openvinotoolkit/openvino that referenced this pull request Jul 23, 2025
### Details:
 - *Separated Vocab MatMul into additional model*
 - *Called infer for additional model in LLMInferRequest*

### Dependencies:
- *OpenVINO GenAI PR:
openvinotoolkit/openvino.genai#2317

### Tickets:
 - *EISW-165218*
@dmatveev dmatveev added this to the 2025.3 milestone Jul 23, 2025
@dmatveev
Copy link
Contributor

dmatveev commented Jul 23, 2025

This PR needs to be updated now I believe

github-merge-queue bot pushed a commit to openvinotoolkit/openvino that referenced this pull request Jul 25, 2025
### Details:
- *Separated Vocab MatMul into additional model*
- *Called infer for additional model in LLMInferRequest*

### Dependencies:
- *OpenVINO GenAI PR* :
openvinotoolkit/openvino.genai#2317

### Tickets:
 - *EISW-165218*

### Note:
- *Regression was found with the PR in the previous merge, PR was
reverted, this regression got fixed in current PR*

---------

Co-authored-by: Dmitry Matveev <dmitry.matveev@intel.com>
@AsyaPronina AsyaPronina force-pushed the three_model_pipeline branch from ee9a7fe to a70cdb1 Compare July 31, 2025 10:59
@AsyaPronina AsyaPronina changed the title LLMPipeline(NPU): Configuration section to fine-tune tail LLMPipeline(NPU): Configuration section to fine-tune LM Head Jul 31, 2025
@AsyaPronina AsyaPronina changed the title LLMPipeline(NPU): Configuration section to fine-tune LM Head LLMPipeline(NPU): Configuration section to fine-tune LM head model Jul 31, 2025
@dmatveev dmatveev self-assigned this Jul 31, 2025
Comment on lines 89 to 90
// Temporarily, for GenAI testing in CI, original update will be in OpenVINO:
update_config(config, {"NPUW_LLM_SHARED_HEAD", "YES"});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this particular line be merged here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, I will remove it!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks!

@Wovchena Wovchena enabled auto-merge August 11, 2025 11:37
@Wovchena Wovchena added this pull request to the merge queue Aug 11, 2025
Merged via the queue into openvinotoolkit:master with commit 5357656 Aug 11, 2025
103 of 106 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants