pkg_resources: do not call stat() and access() #1135
Conversation
| path_item, metadata=PathMetadata( | ||
| path_item, os.path.join(path_item, 'EGG-INFO') | ||
| ) | ||
| if _is_unpacked_egg(path_item): |
There was a problem hiding this comment.
If the code heads down this path now, but path_item is not a dir or is not readable, you'll get a different result than before. That seems to be a backward-incompatible change. Is this code path not expected to be impacted by this change?
There was a problem hiding this comment.
You're right, it probably needs some more work to, I'll dig into it.
There was a problem hiding this comment.
Actually… the code check that path_item/EGG-INFO/PKG-INFO is a file, so if that's the case, path_item has to be a directory. So the check seems useless.
The permission is not that useful either: the fact that the directory is readable is only useful when calling os.listdir, but no code in that path uses os.listdir – and if it does, it'd needs to learn to handle errors. What is interesting is to be sure the directory has +x permission, and that is done by using os.path.isfile in _is_unpacked_egg: it'll return False if it can't traverse the directory.
So… I can't see of a case where things would go wrong. If there's any we should definitely add a test!
|
I like your instinct here. I'm concerned about the test failure on Windows, which does seem implicated in the change. What's your thought? |
|
Interesting… The Windows failure is exactly why Python 3 has a |
The current code in find_on_path is doing a lot of stat() calls which are actually useless and prone to race conditions. As described in Python documentation (https://docs.python.org/3/library/os.html#os.access), os.access must not be used before opening a file. Same goes for a directory. This patch removes those checks by handling exceptions correctly when using os.listdir() instead, which improves pkg_resources import time.
|
Pull-request updated, HEAD is now df23966 |
The current code in find_on_path is doing a lot of stat() calls which are
actually useless and prone to race conditions.
As described in Python documentation
(https://docs.python.org/3/library/os.html#os.access), os.access must not be
used before opening a file. Same goes for a directory.
This patch removes those checks by handling exceptions correctly when using
os.listdir() instead, which improves pkg_resources import time.