New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[microNPU][ETHOSU] Add fixed point for matmul #16401
[microNPU][ETHOSU] Add fixed point for matmul #16401
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR @Aleksei-grovety, I had a couple of questions about the implementation
2983626
to
cb2faa8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the responses @Aleksei-grovety. My main concern is that we need to expose an option to the user, rather than encapsulating this information as part of a relay operator. Are you able to share any more context about where this type of operation comes from?
One of the earliest uses of fixed-point arithmetic was in digital signal processing (DSP) applications, where it was used to perform high-speed and high-precision calculations on audio and video signals. Fixed-point arithmetic was well-suited for these applications because it could provide the necessary precision and speed while minimizing the resources required to perform the calculations. |
69f3b37
to
53d7a2d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the cleanup @Aleksei-grovety. I think overall I'm still a bit unsure why fixed_point_fraction_size
needs to be exposed from the microNPU compiler attrs, rather than the data being encoded in the Relay graph. For example, is it possible to insert relay.fixed_point_multiply
operators on the inputs/outputs of the graph in between the matmul? Apologies if I'm still missing something here
53d7a2d
to
a23e8fc
Compare
Yes, it's possible, I've updated the code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Aleksei-grovety, LGTM!
Add support for calculation matmul with 16 bits fixed point. Add flag enable_fixed_point to enable fixed point calculation.
a23e8fc
to
5e9e432
Compare
Thanks @Aleksei-grovety! |
Added support for calculation matmul with 16 bits fixed point. Added legalization of non-quantized dense operation with quantization by fixed point multiplication.
cc @lhutton1, @ekalda, @leandron