Skip to content

feat(test): add leaderboard#1580

Merged
leshy merged 1 commit intodevfrom
paul/feat/test-speed
Mar 19, 2026
Merged

feat(test): add leaderboard#1580
leshy merged 1 commit intodevfrom
paul/feat/test-speed

Conversation

@paul-nechifor
Copy link
Contributor

Problem

Closes DIM-XXX

Solution

Breaking Changes

How to Test

Contributor License Agreement

  • I have read and approved the CLA.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 16, 2026

Greptile Summary

Adds a new bin/test-speed-leaderboard script that runs pytest, collects test durations from JUnit XML, then uses git blame on each test function's source to attribute total test time to individual committers — producing a ranked leaderboard sorted by time-per-line.

  • The script follows the same patterns as the existing bin/coverage-by-author tool (same git blame porcelain parsing, same repo root detection).
  • Uses AST parsing and caching for efficient function line resolution, and blame caching to avoid redundant git calls.
  • Minor issue: ast.walk in find_function_lines may match class methods when looking for module-level test functions.

Confidence Score: 4/5

  • This PR is safe to merge — it adds a standalone developer utility script with no production code impact.
  • Single new file that is a developer-only CLI tool (bin script). Follows existing codebase conventions closely (mirrors bin/coverage-by-author patterns). No production code is modified. One minor AST traversal issue noted but unlikely to cause practical problems.
  • No files require special attention.

Important Files Changed

Filename Overview
bin/test-speed-leaderboard New 220-line Python script that runs pytest, parses JUnit XML, resolves test functions via AST, and uses git blame to attribute test durations to committers. Well-structured with caching; minor AST walk issue when class_name is None.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Run pytest with JUnit XML output] --> B[Parse JUnit XML for test cases]
    B --> C[For each test case]
    C --> D[Convert classname to file path]
    D --> E[AST parse source file & find function lines]
    E --> F[git blame on function line range]
    F --> G[Attribute duration proportionally by author line count]
    G --> C
    C --> H[Sort authors by time-per-line]
    H --> I[Print ranked leaderboard table]
Loading

Last reviewed commit: 4aa2910

Comment on lines +79 to +87
for node in ast.walk(tree):
if class_name and isinstance(node, ast.ClassDef) and node.name == class_name:
for item in node.body:
if isinstance(item, (ast.FunctionDef, ast.AsyncFunctionDef)):
if item.name == func_name:
return item.lineno, item.end_lineno
elif not class_name and isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if node.name == func_name:
return node.lineno, node.end_lineno
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ast.walk may match class methods for module-level tests

When class_name is None (i.e. the test is a module-level function), ast.walk(tree) traverses every node in the AST including methods inside classes. If a class method happens to share the same name as a module-level test function, this will return whichever one ast.walk yields first — which has no guaranteed order. This could attribute the wrong line range to a test.

Consider restricting the no-class branch to only search tree.body (top-level statements) instead of walking the full tree:

Suggested change
for node in ast.walk(tree):
if class_name and isinstance(node, ast.ClassDef) and node.name == class_name:
for item in node.body:
if isinstance(item, (ast.FunctionDef, ast.AsyncFunctionDef)):
if item.name == func_name:
return item.lineno, item.end_lineno
elif not class_name and isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if node.name == func_name:
return node.lineno, node.end_lineno
def find_function_lines(tree, class_name, func_name):
"""Return (start_line, end_line) of a test function in an AST."""
for node in ast.walk(tree):
if class_name and isinstance(node, ast.ClassDef) and node.name == class_name:
for item in node.body:
if isinstance(item, (ast.FunctionDef, ast.AsyncFunctionDef)):
if item.name == func_name:
return item.lineno, item.end_lineno
if not class_name:
for node in tree.body:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if node.name == func_name:
return node.lineno, node.end_lineno
return None, None

Copy link
Contributor

@leshy leshy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved without a review this is a fun helper

@leshy leshy merged commit a0f0fa7 into dev Mar 19, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants