Skip to content

Commit

Permalink
Deduplicate module file paths to prevent redundant scans.
Browse files Browse the repository at this point in the history
Closes pylint-dev#6242
May also address pylint-dev#4053
  • Loading branch information
emcd committed Nov 11, 2022
1 parent c289271 commit 9e345ba
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 1 deletion.
5 changes: 5 additions & 0 deletions doc/whatsnew/fragments/6242.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Maintain and reference a set of visited files while expanding the module files
from list of module directories, package names, and standalone files to
prevent duplicate scans.

Closes #6242
6 changes: 5 additions & 1 deletion pylint/lint/pylinter.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
import traceback
import warnings
from collections import defaultdict
from collections.abc import Callable, Iterator, Sequence
from collections.abc import Callable, Iterator, MutableSet, Sequence
from io import TextIOWrapper
from pathlib import Path
from re import Pattern
Expand Down Expand Up @@ -878,8 +878,12 @@ def _iterate_file_descrs(
The returned generator yield one item for each Python module that should be linted.
"""
seen_filepaths: MutableSet[str] = set() # For deduplication of paths.
for descr in self._expand_files(files_or_modules):
name, filepath, is_arg = descr["name"], descr["path"], descr["isarg"]
if filepath in seen_filepaths:
continue
seen_filepaths.add(filepath)
if self.should_analyze_file(name, filepath, is_argument=is_arg):
yield FileItem(name, filepath, descr["basename"])

Expand Down

0 comments on commit 9e345ba

Please sign in to comment.