Update quantization overview for XPU #40331
Conversation
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
|
run-slow: aqlm_integration |
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
tests/quantization/ggml/test_ggml.py
Outdated
| out = model.generate(**text, max_new_tokens=10) | ||
|
|
||
| EXPECTED_TEXT = "Hello, I am a 20 year old male" | ||
| EXPECTED_TEXT = "Helloab, I am a 1000" |
There was a problem hiding this comment.
is this th expected output on cpu ?
There was a problem hiding this comment.
I got this output on the Intel Xeon CPU. The new ground truth does not very reaonsble, but you can see otehr gound truth have same issue, like here
There was a problem hiding this comment.
i don't think it's reasonable, we need take a look
There was a problem hiding this comment.
OK, I've reverted this change. Will track it in a separate issue.
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
MekkCyber
left a comment
MekkCyber
left a comment
There was a problem hiding this comment.
LGTM thanks for the update ! Only a small question
| EXPECTED_OUTPUTS = [ | ||
| "Hello my name is John, I am a professional photographer, I", # CUDA output | ||
| "Hello my name is Nils, I am a student of the University", # XPU output | ||
| ] |
There was a problem hiding this comment.
is this the only test where outputs differ ?
There was a problem hiding this comment.
Yes, that's mainly because XPU enables a new way in this PR 395
|
[For maintainers] Suggested jobs to run (before merge) run-slow: aqlm_integration, ggml, quanto_integration |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
6 participants
Update quantization overview for XPU.
Keep in draft until optimum-quanto PR: 395 merged.