Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CPU] Transpose constant folding on cpu plug-in side for MatMul node only #18877

Merged
merged 9 commits into from Aug 7, 2023
Merged

[CPU] Transpose constant folding on cpu plug-in side for MatMul node only #18877

merged 9 commits into from Aug 7, 2023

Conversation

antonvor
Copy link
Contributor

@antonvor antonvor commented Jul 31, 2023 •
edited

This PR moves constant folding of Transpose nodes (only on weights path of FullyConnected node) to the CPU plugin side, that allows to reduce memory consumption of compile model stage

  • added FuseFCAndTransposeOnWeights transformation in cpu graph optimizer
  • fixed RTTI for AlignEltwiseInputRanks and MatMulConstTransposesExtraction transformation passes

@antonvor antonvor self-assigned this Jul 31, 2023
@github-actions github-actions bot added category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations labels Jul 31, 2023
@dmitry-gorokhov dmitry-gorokhov added this to the 2023.1 milestone Aug 1, 2023
@antonvor antonvor marked this pull request as ready for review August 2, 2023 19:19
@github-actions github-actions bot removed the category: transformations OpenVINO Runtime library - Transformations label Aug 2, 2023
@antonvor antonvor requested review from a team as code owners August 2, 2023 19:19
@antonvor antonvor changed the title Feature/f16 transpose cf [CPU] Transpose constant folding on cpu plug-in side for MatMul node only Aug 2, 2023
@antonvor
Copy link
Contributor Author

antonvor commented Aug 3, 2023

@dmitry-gorokhov could you start review please?

@antonvor antonvor requested a review from a team as a code owner August 4, 2023 08:57
@github-actions github-actions bot added the category: transformations OpenVINO Runtime library - Transformations label Aug 4, 2023
dmitry-gorokhov
dmitry-gorokhov reviewed Aug 4, 2023
View reviewed changes
src/plugins/intel_cpu/src/node.h Outdated Show resolved Hide resolved
src/plugins/intel_cpu/src/nodes/fullyconnected.cpp Outdated Show resolved Hide resolved
src/plugins/intel_cpu/src/graph_optimizer.cpp Outdated Show resolved Hide resolved
src/plugins/intel_cpu/src/graph_optimizer.cpp Outdated Show resolved Hide resolved
@dmitry-gorokhov
Copy link
Contributor

dmitry-gorokhov commented Aug 7, 2023 •
edited

Approved as current soltuion. However we will need to follow-up with more scalable approach:
Current design assumes introduction of transposed_weghts to FC operation. which is extension of operation semantics and, in fact, should be treated as workaround.
Proper solution is to change Tranpose output layout to ba and correctly handle it on FC level by utilizing PortDescriptors mechanics.

@moslex moslex added the Feature Freeze Label to highlight PR for Feature Freeze label Aug 7, 2023
@dmitry-gorokhov dmitry-gorokhov merged commit eacdc24 into openvinotoolkit:master Aug 7, 2023
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmitry-gorokhov dmitry-gorokhov dmitry-gorokhov approved these changes

Assignees

@antonvor antonvor

Labels
category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations Feature Freeze Label to highlight PR for Feature Freeze
Projects
None yet
Milestone
2023.1
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@antonvor @dmitry-gorokhov @moslex