for git_file in git_files:
print("walking: ", os.path.join(root, git_file))
run this script:
#!/bin/bashset -xeuo pipefail
rm -rf isort-repro/
git init isort-repro
mkdir build/ src/
touch build/1.py build/2.py build/3.py # these should be ignored
touch src/main.py # this should be included
isort src/ --skip-gitignore
Hello @bmalehorn thanks for such a detailed ticket!
To be honest, I don't have much context on why this was done like this but if I have to argue for the present implementation it would be that get_files is a generic function which is being used with all configuration settings with _check_git_ignore being the extra check performed specifically for the skip-ignore flag. This seems to me a better design paradigm with base functions being used in the general case augmented with particular functionality when needed. The instance you shared with 8 million files will fall in the minority and then again I'll guess that symlinks would constitute a major chunk of these files which could be avoided using the dont-follow-links flag. Not to mention the extra dependency of the user having git installed in the system for git ls-files to work without which isort would anyways have to revert back to os.walk
Please raise if there's anything I might have gotten wrong and I would love to discuss this further and hear any other perspectives on this.
Hi, I'm looking into a performance issue with isort.
It seems that when you pass in
--skip-ignoreand a directory, it will:
Lines 555 to 586 in c6a4196
In my use case, my subdirectory has 8 files. But before checking those, isort will first enumerate all 8 million untracked files in the entire repo, which takes several minutes!
There are a couple potential fixes:
os.walk, and instead run
git ls-files. This gets all tracked files (which there's a lot less of) instead of getting all untracked files and filtering
os.walkin the subdirectory you're running in, instead of the entire git repo
I'd be happy to prep a PR fixing this issue, but I want to see if that makes sense or if there's other considerations I'm not taking into account.
or maybe even this, since I only passed in
These PRs are related to inclusion rules, but are specifically about symlinks: #1772, #1788
The text was updated successfully, but these errors were encountered: