Add an "onitem" callback parameter to shutil.rmtree()

[websurfer5]

BPO	37366
Nosy	@pfmoore, @giampaolo, @tiran, @tjguk, @tarekziade, @pkch, @zware, @serhiy-storchaka, @zooba, @MojoVampire, @websurfer5
PRs	bpo-37366: add onitem callback argument to shutil.rmtree() #14419

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-06-29.12:00:11.868>
created_at = <Date 2019-06-21.19:25:52.405>
labels = ['3.7', '3.8', 'type-feature', 'library', '3.9']
title = 'Add an "onitem" callback parameter to shutil.rmtree()'
updated_at = <Date 2019-07-03.20:08:37.744>
user = 'https://github.com/websurfer5'

bugs.python.org fields:

activity = <Date 2019-07-03.20:08:37.744>
actor = 'Jeffrey.Kintscher'
assignee = 'none'
closed = True
closed_date = <Date 2019-06-29.12:00:11.868>
closer = 'giampaolo.rodola'
components = ['Library (Lib)']
creation = <Date 2019-06-21.19:25:52.405>
creator = 'Jeffrey.Kintscher'
dependencies = []
files = []
hgrepos = []
issue_num = 37366
keywords = ['patch']
message_count = 5.0
messages = ['346245', '346719', '346805', '346806', '346879']
nosy_count = 12.0
nosy_names = ['paul.moore', 'giampaolo.rodola', 'christian.heimes', 'tim.golden', 'tarek', 'max', 'zach.ware', 'serhiy.storchaka', 'riccardomurri', 'steve.dower', 'josh.r', 'Jeffrey.Kintscher']
pr_nums = ['14419']
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue37366'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[websurfer5]

Add an "onitem" callback paramter to shutil.rmtree() that, if provided, gets called for each directory entry as it is encountered. This allows the caller to perform any required special handling of individual directory entries (e.g. unmounting a mount point, closing a file, shutting down a named pipe, etc.) before rmtree() attempts to remove them.

This enhancement is related to issue bpo-36422.

[giampaolo]

I am not sure I understand the use case. Other than "unmounting a mount point" the other scenarios look unrealistic to me and also require a fd instead of a path. Also, if you want to unmount a path you can do so via *onerror* instead (unmount + delete). Pre-emptively checking if you have the permission to do something is sort of unpythonic and racy.

[websurfer5]

Yes, onerror allows you to delete an undeletable item. But, in the case of a mount point, the contents of the mounted filesystem get deleted *before* the mount point triggers onerror. This is the case described in issue bpo-36422.

The behavior provided by onitem allows you to provide a callback that can check for specific paths that may require special handling. This is especially useful if the onitem callback is also added to tempfile.TemporaryDirectory().

My PR handles a missing onitem argument the same way onerror is handled by providing a do-nothing stub function:

if onitem is None:
    def onitem(*args):
        pass

I don't know enough about CPython internals to tell the performance differences between such a stub function and wrapping each call to onitem in an "if" block.

[tiran]

I'm with Giampaolo.

I'm also concerned that the new callback will make the rmtree function too complicated. It's not a one-size-fits-all solution, but it covers most uses cases.