Skip to content

Conversation

liulei08
Copy link
Contributor

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

修复swift eval参数dataset_args为eval_dataset_args,与训练过程中eval的对应参数保持一致。
对应的issue:#5968

Experiment results

Paste your experiment result here(if needed).

Copy link
Contributor

Summary of Changes

Hello @liulei08, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a bug related to inconsistent parameter naming within the evaluation module. It renames the dataset_args parameter to eval_dataset_args across the EvalArguments class definition, its initialization, and its usage in the evaluation logic. This change ensures a unified naming scheme, aligning evaluation parameters with existing training parameter conventions, which enhances code readability and maintainability.

Highlights

  • Parameter Renaming: The dataset_args field within the EvalArguments class in swift/llm/argument/eval_args.py has been renamed to eval_dataset_args for improved clarity and consistency.
  • Argument Parsing Update: The __post_init__ method in EvalArguments was updated to correctly process the newly named eval_dataset_args when parsing arguments.
  • Evaluation Logic Alignment: The get_native_task_cfg function in swift/llm/eval/eval.py was modified to pass the args.eval_dataset_args parameter, ensuring the evaluation logic uses the consistent naming convention.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

本次 PR 旨在修复 swift eval 中的参数命名不一致问题,将 dataset_args 重命名为 eval_dataset_args,使其与训练过程中的参数保持一致。代码变更简单明了,逻辑正确。我唯一的建议是为这个新参数补充相应的测试用例,以保证其功能的稳定性和正确性。

eval_dataset: List[str] = field(default_factory=list)
eval_limit: Optional[int] = None
dataset_args: Optional[Union[Dict, str]] = None
eval_dataset_args: Optional[Union[Dict, str]] = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

参数已成功重命名为 eval_dataset_args 以提高一致性。为了确保此参数的正确功能并防止未来出现回归问题,建议在 tests/eval/test_eval.py 中添加一个测试用例来覆盖 eval_dataset_args 的使用场景。例如,可以在 test_eval_native 中加入 eval_dataset_args

@Jintao-Huang
Copy link
Collaborator

方便把文档也修改一下不

然后加一下 dataset_args 的向后兼容,默认为None,如果设置了,就覆盖eval_dataset_args

@liulei08
Copy link
Contributor Author

方便把文档也修改一下不

然后加一下 dataset_args 的向后兼容,默认为None,如果设置了,就覆盖eval_dataset_args

好的,文档我也改一下

@Jintao-Huang Jintao-Huang merged commit 315f2c6 into modelscope:main Oct 2, 2025
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants