Skip to content

[fix] swift eval parameter dataset_args is replaced by eval_dataset_args #5969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 2, 2025
Merged

[fix] swift eval parameter dataset_args is replaced by eval_dataset_args #5969

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f83532d
修复Windows环境下转换json字典字符串异常
liulei08 Sep 14, 2025
7f672f8
修复Windows环境下转换json字典字符串异常
liulei08 Sep 19, 2025
4412c4f
修复Windows环境下转换json字典字符串异常
liulei08 Sep 19, 2025
9ab699f
修复eval参数dataset_args为eval_dataset_args
liulei08 Sep 25, 2025
0e518dd
Merge branch 'modelscope:main' into main
liulei08 Sep 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions swift/llm/argument/eval_args.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ class EvalArguments(DeployArguments):
"""
eval_dataset: List[str] = field(default_factory=list)
eval_limit: Optional[int] = None
dataset_args: Optional[Union[Dict, str]] = None
eval_dataset_args: Optional[Union[Dict, str]] = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

参数已成功重命名为 eval_dataset_args 以提高一致性。为了确保此参数的正确功能并防止未来出现回归问题，建议在 tests/eval/test_eval.py 中添加一个测试用例来覆盖 eval_dataset_args 的使用场景。例如，可以在 test_eval_native 中加入 eval_dataset_args。

eval_generation_config: Optional[Union[Dict, str]] = field(default_factory=dict)
eval_output_dir: str = 'eval_output'
eval_backend: Literal['Native', 'OpenCompass', 'VLMEvalKit'] = 'Native'
Expand All @@ -48,7 +48,7 @@ def __post_init__(self):
super().__post_init__()
self._init_eval_url()
self._init_eval_dataset()
self.dataset_args = json_parse_to_dict(self.dataset_args)
self.eval_dataset_args = json_parse_to_dict(self.eval_dataset_args)
self.eval_generation_config = json_parse_to_dict(self.eval_generation_config)
self.extra_eval_args = json_parse_to_dict(self.extra_eval_args)
self.eval_output_dir = to_abspath(self.eval_output_dir)
Expand Down
2 changes: 1 addition & 1 deletion swift/llm/eval/eval.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def get_native_task_cfg(self, dataset: List[str], url: str):
work_dir=work_dir,
limit=args.eval_limit,
eval_batch_size=args.eval_num_proc,
dataset_args=args.dataset_args,
dataset_args=args.eval_dataset_args,
generation_config=args.eval_generation_config,
**args.extra_eval_args)

Expand Down
Loading