You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
assignee=Noneclosed_at=<Date2017-11-04.12:46:52.211>created_at=<Date2016-10-30.20:02:39.210>labels= ['3.7', 'type-feature', 'library']
title='shutil.rmtree is inefficient due to listdir() instead of scandir()'updated_at=<Date2017-12-30.05:15:42.120>user='https://github.com/enkore'
The main issue on *nix is more likely that by using listdir you get directory order, while what you really need is inode ordering. scandir allows for that, since you get the inode from the DirEntry with no extra syscalls - especially without an open() or stat().
Other optimizations are also possible. For example opening the directory and using unlinkat() would likely shave off a bit of CPU. But the dominating factor here is likely the bad access pattern.
Following Antoine's suggestion the patch now makes shutil.rmtree() using os.scandir() on all platforms.
I doubt about one thing. This patch changes os.listdir passed to the onerror handler to os.scandir. This can break a user code that checks if the first argument in onerror is os.listdir. If keep this change, it should be documented in the "Porting to 3.7" section. Alternatively, we can continue passing os.listdir if os.scandir() failed despites the fact that os.listdir no longer used.