[XPU][CI]Update test assertion and base response value #4907
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR modifies the test validation logic in the VL (Vision-Language) model test to be more flexible. Instead of requiring an exact match with the expected response, it now checks if the key information is contained within the response.
Key Changes:
- Updated the expected response string to contain only the core information ("北魏时期") instead of the full formatted response
- Changed assertion from exact equality check to substring containment check
| service_http_port = "8188" # 服务配置的 | ||
| client = openai.Client(base_url=f"http://{ip}:{service_http_port}/v1", api_key="EMPTY_API_KEY") | ||
| base_response = "\n\n这尊佛像属于**北魏时期**" | ||
| base_response = "北魏时期" |
There was a problem hiding this comment.
变量名 base_response 不够准确。既然现在它只包含关键信息而不是完整响应,建议重命名为 expected_keyword 或 expected_content 以更好地反映其用途。
| print(response.choices[0].message.content) | ||
| # print(base_response) | ||
| assert response.choices[0].message.content == base_response | ||
| assert base_response in response.choices[0].message.content |
There was a problem hiding this comment.
断言缺少错误消息。建议添加描述性错误消息以便在测试失败时更容易调试,例如:assert base_response in response.choices[0].message.content, f\"Expected '{base_response}' not found in response: {response.choices[0].message.content}\"
EmmonsCurse
approved these changes
Nov 10, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.