[UT]: add UT coverage for swebench#316
Conversation
- Add test_swebench.py with 12 test cases covering all code paths - Test valid aggregate results, zero total, negative total - Test missing files, invalid data, load exceptions - Test empty configs, string total instances, etc. - Achieves 100% line coverage for swebench.py
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive suite of unit tests for the SWEBenchSummarizer class in tests/UT/summarizers/test_swebench.py, covering various edge cases for the _pick_up_results method. The review feedback suggests several improvements: removing the unused MagicMock import, targeting mocks locally within the module under test rather than globally on os.path and mmengine to prevent side effects, and adding a trailing newline at the end of the file to adhere to PEP 8 standards.
| import os | ||
| import tempfile | ||
| import shutil | ||
| from unittest.mock import patch, MagicMock |
| @patch('os.path.isdir') | ||
| @patch('os.path.isfile') | ||
| @patch('mmengine.load') | ||
| @patch('ais_bench.benchmark.summarizers.swebench.model_abbr_from_cfg_used_in_summarizer') | ||
| @patch('ais_bench.benchmark.summarizers.swebench.dataset_abbr_from_cfg') | ||
| @patch('ais_bench.benchmark.summarizers.swebench.get_infer_output_path') |
There was a problem hiding this comment.
建议:将全局的 os.path 和 mmengine 补丁(patch)替换为局部模块补丁。
原因:
直接对 os.path.isdir、os.path.isfile 和 mmengine.load 进行全局 Mock 会污染全局命名空间。如果在测试运行期间,测试框架或其他标准库函数(例如 shutil.rmtree 或 tempfile 相关操作)调用了这些方法,可能会导致不可预测的副作用或测试套件不稳定。
根据 Python unittest.mock 的最佳实践,应该在使用该函数的地方(即目标模块 ais_bench.benchmark.summarizers.swebench)进行 Mock,而不是在其定义的地方。
改进方案:
将 os.path.isdir 替换为 ais_bench.benchmark.summarizers.swebench.osp.isdir,将 os.path.isfile 替换为 ais_bench.benchmark.summarizers.swebench.osp.isfile,将 mmengine.load 替换为 ais_bench.benchmark.summarizers.swebench.mmengine.load。
(注:此建议同样适用于文件中其他测试用例的 @patch 装饰器)
| @patch('os.path.isdir') | |
| @patch('os.path.isfile') | |
| @patch('mmengine.load') | |
| @patch('ais_bench.benchmark.summarizers.swebench.model_abbr_from_cfg_used_in_summarizer') | |
| @patch('ais_bench.benchmark.summarizers.swebench.dataset_abbr_from_cfg') | |
| @patch('ais_bench.benchmark.summarizers.swebench.get_infer_output_path') | |
| @patch('ais_bench.benchmark.summarizers.swebench.osp.isdir') | |
| @patch('ais_bench.benchmark.summarizers.swebench.osp.isfile') | |
| @patch('ais_bench.benchmark.summarizers.swebench.mmengine.load') | |
| @patch('ais_bench.benchmark.summarizers.swebench.model_abbr_from_cfg_used_in_summarizer') | |
| @patch('ais_bench.benchmark.summarizers.swebench.dataset_abbr_from_cfg') | |
| @patch('ais_bench.benchmark.summarizers.swebench.get_infer_output_path') |
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| unittest.main() No newline at end of file |
概述
为
ais_bench/benchmark/summarizers/swebench.py模块添加单元测试,实现 100% 代码覆盖率。变更内容
新增文件
tests/UT/summarizers/test_swebench.py- SWEBenchSummarizer 单元测试测试用例 (12 个)
test_pick_up_results_valid_aggregatetest_pick_up_results_no_dirtest_pick_up_results_no_aggregate_filetest_pick_up_results_zero_total_instancestest_pick_up_results_invalid_datatest_pick_up_results_missing_fieldstest_pick_up_results_negative_totaltest_pick_up_results_load_exceptiontest_pick_up_results_100_percenttest_pick_up_results_string_total_instancestest_pick_up_results_empty_configstest_pick_up_results_resolved_greater_than_totalUT测试用例通过情况
全量覆盖率