Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use scandir() in find_commands.py to avoid stat() calls against 23.7.x branch #13071

Merged
merged 1 commit into from
Sep 11, 2023

Conversation

dholth
Copy link
Contributor

@dholth dholth commented Sep 7, 2023

Description

Fixes #13033, #13067

The original fix from PR #13035 is already in the main branch but not in the 23.7.x branch. This fix improves on #13035 by also removing the os.path.isfile() call.

In general, we should always use scandir(); I didn't notice any equally problematic listdir()/stat() pairs on a quick search through the conda source.

Checklist - did you ...

  • Add a file to the news directory (using the template) for the next release's release notes?
  • Add / update necessary tests?
  • Add / update outdated documentation?

@dholth dholth requested a review from a team as a code owner September 7, 2023 15:09
@conda-bot conda-bot added the cla-signed [bot] added once the contributor has signed the CLA label Sep 7, 2023
@dholth
Copy link
Contributor Author

dholth commented Sep 7, 2023

I tried this on wsl2. I installed miniconda3 including conda 23.5, and "conda info" was fast. I used conda to update itself, and conda startup was slow. I did not wait for it to finish starting. Then I checked out the main branch, and these fixes both against main or 23.7.x.

On the main branch with only the older fix conda was fast again. On my machine scandir() avoids 4 stat calls since there are 4 conda-* files in PATH.

This newer fix is not measurably different than the existing fix since it would only avoid 4 stat calls. But with scandir() it is still fast even if we call entry.is_file() before checking pat.match(entry.name). We should keep the current "match first" order to avoid following non-conda-* symlinks.

In the future we could even consider calling os.scandir(dir_path) without checking if not isdir(dir_path) first and catch FileNotFoundError / NotADirectoryError.

I also was able to run "conda content-trust" which AFAICT is not a new-style plugin in conda-content-trust=0.1.3 as installed in the test environment.

@dholth dholth enabled auto-merge (squash) September 7, 2023 16:33
@jezdez jezdez mentioned this pull request Sep 11, 2023
62 tasks
@kenodegard kenodegard merged commit 34c6746 into 23.7.x Sep 11, 2023
63 of 69 checks passed
@kenodegard kenodegard deleted the 23.7.x-find-commands-scandir branch September 11, 2023 15:11
@kenodegard kenodegard linked an issue Sep 11, 2023 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed [bot] added once the contributor has signed the CLA
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Conda 23.7.3 start-up time extremely slow compared to 23.5.2
5 participants