pycore: don't stop source folder detection at the first package - #869
Open
quazardous wants to merge 2 commits into
Open
pycore: don't stop source folder detection at the first package#869quazardous wants to merge 2 commits into
quazardous wants to merge 2 commits into
Conversation
`_find_source_folders` returned as soon as a folder had a package among its children, leaving every sibling folder unscanned. On a project whose root holds a package, detection therefore stopped at the root and no other source folder was ever found. That contradicts what the existing tests describe. Both `test_multi_source_folders` and `test_multi_source_folders2` expect two roots side by side -- they only pass because in their layouts no package sits directly at the root, so the early return never fires. The consequence is silent. Imports resolving through the unscanned folders simply do not resolve, so rename skips those occurrences instead of reporting them: a repository-wide rename comes back having quietly renamed a subset. Apply the same rule at every level instead of stopping at the first match, and skip package subfolders when recursing -- descending into a package would make `package.sub` importable as plain `sub`. Measured on a project with two packages at the root and further code under `src/`: 1 source folder detected before, 15 after, and an occurrence that resolved 0 of its 2 references now resolves both. This costs time where it previously skipped work. The same rename went from 3.5s to 43.8s, and touches 19 files rather than 8 -- the added time is spent analysing code that was invisible before, not overhead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V5GqJKaXZzFzmFEYgo4pPR
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V5GqJKaXZzFzmFEYgo4pPR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect
_find_source_foldersreturns as soon as a folder has a package among its children:So on a project whose root holds a package, detection stops at the root and no other source folder is ever found.
Why this looks like a defect rather than a design choice
Two existing tests already describe side-by-side roots:
test_multi_source_folderssrcandtesttest_multi_source_folders2srcBoth pass today only because in their layouts no package sits directly at the root, so the early return never fires. Multi-root detection is clearly intended; the early return silently disables it for any repository that grew a package at its top level.
Why it matters
The consequence is silent, which is the worst part. Imports that would resolve through the unscanned folders simply do not resolve, so
Renameskips those occurrences rather than reporting them — a repository-wide rename comes back having quietly renamed a subset. There is no error and nounsureoccurrence to review.The change
Apply the same rule at every level instead of stopping at the first match, and skip package subfolders when recursing — descending into a package would make
package.subimportable as plainsub.Two tests added. The first fails on
master; the second guards the subpackage boundary and passes either way.Measurements
On a real project whose root holds two packages with further code under
src/:from pkg import modsrc/pkg/mod.pyropetest/: 2122 passed, 7 skipped, 5 xfailed.On the cost, plainly: this spends time where detection previously skipped work, so a project that gains source folders pays for analysing code that was invisible before. It is not overhead, but it is real, and on a project where the old single root was the right answer there is nothing to gain and time to lose.
source_foldersin the prefs still overrides detection for anyone who wants to narrow it back.I measured a second case where the coverage actually changes — an occurrence resolving 0 of 2 references before and 2 of 2 after — but that rename needs #868 to run at all on this codebase, so I am not claiming it for this PR alone.
Relation to #868
Independent, and based on
master. #868 fixes the walker matching tokens inside string literals; this one fixes which folders are searched. Either can land first.🤖 Generated with Claude Code
https://claude.ai/code/session_01V5GqJKaXZzFzmFEYgo4pPR