[XPU][CI]Change W4A8 Case Base Value#5309
Conversation
|
Thanks for your contribution! |
There was a problem hiding this comment.
Pull request overview
This PR updates the XPU CI test for W4A8 quantization by expanding the acceptable response keywords in the assertion. The change adds "小度" (Xiaodu) as a valid keyword alongside the existing "人工智能" (artificial intelligence) and "文心一言" (ERNIE Bot) keywords when validating the model's self-introduction response.
Key Changes
- Expanded the keyword validation list in the W4A8 quantization test to include "小度" as an acceptable model identity response
| print(response.choices[0].message.content) | ||
| # print(base_response) | ||
| assert any(keyword in response.choices[0].message.content for keyword in ["人工智能", "文心一言"]) | ||
| assert any(keyword in response.choices[0].message.content for keyword in ["人工智能", "文心一言", "小度"]) |
There was a problem hiding this comment.
此PR的描述信息不完整。虽然标题说明了要更改基准值，但PR描述中的"Motivation"和"Modifications"部分都是空的，没有解释为什么要添加"小度"作为可接受的关键词。
建议在PR描述中说明：
- Motivation: 为什么需要添加"小度"作为可接受的响应关键词？是因为模型响应行为发生了变化，还是为了支持新的模型版本？
- Modifications: 明确说明在w4a8量化测试中扩展了验证关键词列表，从["人工智能", "文心一言"]增加到["人工智能", "文心一言", "小度"]。
- Accuracy Tests: 如果这个改动影响了模型输出验证逻辑，请提供相关的测试结果或示例响应。
这样可以帮助审阅者理解修改的必要性和合理性。
plusNew001
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #5309 +/- ##
==========================================
Coverage ? 59.77%
==========================================
Files ? 324
Lines ? 39598
Branches ? 5954
==========================================
Hits ? 23671
Misses ? 14051
Partials ? 1876
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
2 participants
Motivation
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.