Skip to content

Update result analyzer and remaining changes#23

Open
eerxuan wants to merge 1 commit intodocumentdb:mainfrom
eerxuan:result-analyzer-updates-2
Open

Update result analyzer and remaining changes#23
eerxuan wants to merge 1 commit intodocumentdb:mainfrom
eerxuan:result-analyzer-updates-2

Conversation

@eerxuan
Copy link
Copy Markdown
Contributor

@eerxuan eerxuan commented Mar 31, 2026

Update result_analyzer with improved categorization and reporting.

This has dependency on #20.

Copy link
Copy Markdown

@xgerman xgerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as a human.

AI felt this could be done:
Issues Found

🟡 Medium — Import at function scope inside print_summary() report_generator.py line ~158: from collections
import Counter is imported inside the function body. This should be a top-level import per PEP 8. It works, but
is inconsistent with the rest of the file.

🟡 Medium — failure_type not set for non-FAIL outcomes analyzer.py line ~340-345: failure_type is only added to
the test detail dict when test_outcome == FAIL. But report_generator.py calls test.get("failure_type", "UNKNOWN")
— this works due to .get() with default, but the field's conditional presence is fragile. Consider always setting
it (e.g., "N/A" for passed/skipped).

🟢 Minor — Trailing whitespace cleanup is noise Many hunks are whitespace-only changes (trailing spaces removed).
Not a problem, but inflates the diff. Could have been a separate commit.

🟢 Minor — extract_failure_tag regex is broad

match = re.search(r'[([A-Z_]+)]', crash_message)

This matches any [UPPER_CASE] bracket pattern. If a crash message happens to contain something like
[SOME_BSON_KEY], it could false-match. Not likely in practice but worth noting.

🟢 Minor — Empty init.py tests/collection/init.py is added with just a docstring. Fine for package
recognition, but if PR #20 already added test files under tests/collection/, this should have been part of that
PR.

Update result_analyzer with improved categorization and reporting.

Signed-off-by: Yunxuan Shi <yunxuan@amazon.com>
@eerxuan eerxuan force-pushed the result-analyzer-updates-2 branch from 650a516 to e515b94 Compare April 2, 2026 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants