Skip to content

perf: prefilter files by path before read_text() in discovery#2134

Open
KRRT7 wants to merge 3 commits intomainfrom
perf/discovery-prefilter
Open

perf: prefilter files by path before read_text() in discovery#2134
KRRT7 wants to merge 3 commits intomainfrom
perf/discovery-prefilter

Conversation

@KRRT7
Copy link
Copy Markdown
Collaborator

@KRRT7 KRRT7 commented May 7, 2026

Parent PR

#2132 — perf: targeted performance improvements for E2E pipeline hot path

Summary

  • Moves path-based filtering (test files, ignored paths, submodules, site-packages, outside module-root) to run BEFORE read_text() during function discovery
  • Files that would be discarded by filter_functions() are now skipped before any I/O
  • filter_functions() still runs afterward as a safety net for callers that don't pass prefilter params (e.g. LSP)

Changes

  • get_all_files_and_functions(): added optional tests_root, module_root params to enable prefiltering
  • get_functions_within_lines() / get_functions_within_git_diff(): same optional params
  • New test file: tests/test_discovery_prefilter.py (7 tests)

Stack

  1. This PRfix/mypy-strict-errors
  2. perf: cache file reads and AST parses during discovery pass #2135 (ast-cache) → this branch
  3. perf: read JS/TS files once during discovery export checks #2136 (js-single-read) → ast-cache branch

Moves path-based checks (test file detection, ignore paths, submodule
paths, site-packages, outside module-root) to run BEFORE read_text() is
called in get_all_files_and_functions() and get_functions_within_lines().
This avoids unnecessary file I/O for files that would be discarded by
filter_functions() anyway.

Also fixes pre-existing mypy error in _find_all_functions_via_language_support
where discover_functions was called with wrong argument order.

Signature changes (backward-compatible, all new params are optional):
- get_all_files_and_functions: added tests_root, module_root params
- get_functions_within_lines: added tests_root, ignore_paths, module_root
- get_functions_within_git_diff: added tests_root, ignore_paths, module_root
@KRRT7 KRRT7 force-pushed the fix/mypy-strict-errors branch from 593a5a7 to 4bf0aaf Compare May 7, 2026 22:47
Base automatically changed from fix/mypy-strict-errors to fix/subprocess-xml-errors May 7, 2026 22:47
@KRRT7 KRRT7 force-pushed the perf/discovery-prefilter branch from 6bbf072 to 6527780 Compare May 7, 2026 22:47
@KRRT7 KRRT7 marked this pull request as draft May 7, 2026 22:53
Base automatically changed from fix/subprocess-xml-errors to fix/test-files-silent-dedup May 8, 2026 00:15
@KRRT7 KRRT7 marked this pull request as ready for review May 8, 2026 00:17
@KRRT7 KRRT7 changed the base branch from fix/test-files-silent-dedup to main May 8, 2026 00:18
aseembits93
aseembits93 previously approved these changes May 8, 2026
@aseembits93 aseembits93 enabled auto-merge May 8, 2026 00:26
@KRRT7 KRRT7 dismissed aseembits93’s stale review May 8, 2026 00:26

The merge-base changed after approval.

@KRRT7 KRRT7 requested a review from aseembits93 May 8, 2026 01:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants