WT-12969 Generate per test code coverage results #10638
Conversation
test/evergreen.yml
Outdated
| # Logging for debugging | ||
| ls -l coverage_report | ||
| cat coverage_report/diff.txt | ||
| ${python_binary|python3} test/evergreen/code_change_report/per_test_code_coverage_report.py -v -c coverage_data -d coverage_report/diff.txt -m coverage_report/metrixpp.csv |
There was a problem hiding this comment.
we can safely use python3 here, unless we are running this on a mac.
| if not Path(gcovr_dir).is_absolute(): | ||
| sys.exit("gcovr_dir must be an absolute path") | ||
|
|
||
| build_dirs = check_build_dirs(build_dir_base=build_dir_base, parallel=parallel_tests, setup=setup, verbose=verbose) |
There was a problem hiding this comment.
there is no need for the equal sign, as we can just pass in the argument itself.
There was a problem hiding this comment.
In this case, you're correct so I've removed them. But very useful sometimes.
| with open(config_path) as json_file: | ||
| config = json.load(json_file) | ||
|
|
||
| if verbose: |
There was a problem hiding this comment.
It looks as tho, we would benefit from a logger class. We pass in the verbose variable into the class which would check the if condition instead, such that whenever we do something like:
logger.message("hello world"), it would check the verbose variable if it should actually print to stdout or not.
There was a problem hiding this comment.
Good idea - I've introduced a logger.
| git_diff_file = args.git_diff_file | ||
| complexity_data_file = args.metrix_complexity_data | ||
|
|
||
| if verbose: |
There was a problem hiding this comment.
It looks like this would also benefit from a logger class. The idea is that we put in verbose variable, and the class would check if verbose is set, if so, we would print the message. E.g. (logger.message("hello world"), would not print if verbose is set to false.
There was a problem hiding this comment.
Good idea - I've introduced a logger.
|
Hi @jiechenbo and @ajmorton , I've updated the branch/PR based on Jie's feedback. Please let me know what you think. |
| info = json.load(json_file) | ||
| return info |
There was a problem hiding this comment.
| info = json.load(json_file) | |
| return info | |
| return json.load(json_file) |
| hunks = patch.hunks | ||
| hunk_list = list() | ||
| for hunk in hunks: |
There was a problem hiding this comment.
| hunks = patch.hunks | |
| hunk_list = list() | |
| for hunk in hunks: | |
| hunk_list = list() | |
| for hunk in patch.hunks: |
| change_info = dict() | ||
| change_info['status'] = hunk.status | ||
| change_info['new_start'] = hunk.new_start | ||
| change_info['new_lines'] = hunk.new_lines | ||
| change_info['old_start'] = hunk.old_start | ||
| change_info['old_lines'] = hunk.old_lines |
There was a problem hiding this comment.
| change_info = dict() | |
| change_info['status'] = hunk.status | |
| change_info['new_start'] = hunk.new_start | |
| change_info['new_lines'] = hunk.new_lines | |
| change_info['old_start'] = hunk.old_start | |
| change_info['old_lines'] = hunk.old_lines | |
| change_info = { 'status': hunk.status, 'new_start': hunk.new_start, 'new_lines': hunk.new_lines, 'old_start': hunk.old_start, 'old_lines': hunk.old_lines } |
| line_info = dict() | ||
| line_info['content'] = line.content | ||
| line_info['new_lineno'] = line.new_lineno | ||
| line_info['old_lineno'] = line.old_lineno |
There was a problem hiding this comment.
| line_info = dict() | |
| line_info['content'] = line.content | |
| line_info['new_lineno'] = line.new_lineno | |
| line_info['old_lineno'] = line.old_lineno | |
| line_info = { 'content': line.content, 'new_lineno': line.new_lineno, 'old_lineno': line.old_lineno } |
| diff_file = open(git_diff_file, mode="r") | ||
| diff_data = diff_file.read() | ||
| diff = Diff.parse_diff(diff_data) | ||
| change_list = diff_to_change_list(diff=diff, verbose=verbose) | ||
|
|
||
| complexity_data = read_complexity_data(complexity_data_file) | ||
| preprocessed_complexity_data = preprocess_complexity_data(complexity_data=complexity_data) | ||
|
|
||
| coverage_data = collate_coverage_data(gcovr_dir=coverage_data_path, verbose=verbose) | ||
|
|
||
| generate_report(coverage_data=coverage_data, change_list=change_list, | ||
| preprocessed_complexity_data=preprocessed_complexity_data, verbose=verbose) | ||
|
|
There was a problem hiding this comment.
| diff_file = open(git_diff_file, mode="r") | |
| diff_data = diff_file.read() | |
| diff = Diff.parse_diff(diff_data) | |
| change_list = diff_to_change_list(diff=diff, verbose=verbose) | |
| complexity_data = read_complexity_data(complexity_data_file) | |
| preprocessed_complexity_data = preprocess_complexity_data(complexity_data=complexity_data) | |
| coverage_data = collate_coverage_data(gcovr_dir=coverage_data_path, verbose=verbose) | |
| generate_report(coverage_data=coverage_data, change_list=change_list, | |
| preprocessed_complexity_data=preprocessed_complexity_data, verbose=verbose) | |
| with open(git_diff_file, mode="r") as diff_file: | |
| diff_data = diff_file.read() | |
| diff = Diff.parse_diff(diff_data) | |
| change_list = diff_to_change_list(diff=diff, verbose=verbose) | |
| complexity_data = read_complexity_data(complexity_data_file) | |
| preprocessed_complexity_data = preprocess_complexity_data(complexity_data=complexity_data) | |
| coverage_data = collate_coverage_data(gcovr_dir=coverage_data_path, verbose=verbose) | |
| generate_report(coverage_data=coverage_data, change_list=change_list, | |
| preprocessed_complexity_data=preprocessed_complexity_data, verbose=verbose) | |
Co-authored-by: Andrew Morton <andrew.morton@mongodb.com>
…added more debug output.
This ticket adds Python code and an Evergreen task to generate per-task code coverage results and, for patch builds, create a simple report on which tests reach each added/changed WT function.