shutil.copyfile blocks indefinitely on named pipes

[aioryi]

BPO	3002
Nosy	@pitrou
Files	issue3002.patch issue3002-2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-05-01.21:10:30.657>
created_at = <Date 2008-05-29.15:32:28.978>
labels = ['type-bug', 'library']
title = 'shutil.copyfile blocks indefinitely on named pipes'
updated_at = <Date 2009-05-01.21:10:30.656>
user = 'https://bugs.python.org/aioryi'

bugs.python.org fields:

activity = <Date 2009-05-01.21:10:30.656>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2009-05-01.21:10:30.657>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2008-05-29.15:32:28.978>
creator = 'aioryi'
dependencies = []
files = ['13269', '13270']
hgrepos = []
issue_num = 3002
keywords = ['patch']
message_count = 10.0
messages = ['67498', '67499', '67700', '67758', '67773', '81802', '83312', '83335', '83336', '86910']
nosy_count = 5.0
nosy_names = ['pitrou', 'draghuram', 'schmir', 'aioryi', 'farialima']
pr_nums = []
priority = 'critical'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue3002'
versions = ['Python 2.6', 'Python 2.5', 'Python 3.0']

[aioryi]

shutil.copytree() uses shutil.copyfile() to copy files recursively.
shutil.copyfile() opens the source file for reading, and the destination
file for writing, followed by a call to shutil.copyfileobj().

If the file happens to be a named pipe rather than a normal file,
opening for read blocks the copying function, since the Unix OS needs a
writer process to attach to the same named pipe before the open-for-read
succeeds.

Rather than opening the file for reading, the correct action would
probably be to simply create a new named pipe with the same name at the
destination.
Looking at the Python2.3 code, the same type of problem seem to exist
for other non-normal file-types other than symlinks (eg device files,
sockets, and possibly a few others).

[draghuram]

I am not sure if copyfile() should be trying to copy named pipes (or any
other special files for that matter). The best way is perhaps to check
and skip such files.

[schmir]

I can confirm this issue on python 2.5.
I think the priority should be set to critical, as this can lead to
denial of service attacks.

[schmir]

The open('fifo', 'rb') already blocks. One has to use os.fdopen with
O_NONBLOCK to prevent blocking.

fd=os.open('fifo', os.O_RDONLY | os.O_NONBLOCK)

and then use

stat.S_ISFIFO(os.fstat(fd).st_mode)

to check if this is a fifo.
Checking with os.stat before opening with open will result in a race
condition.

[schmir]

if the destination is a named pipe, it will also block (waiting for a
reader).

[farialima]

Note that the 'cp' user command in Linux blocks also. Although this is
*very* painful, you could argue that since the OS commands do the same
thing, it shouldn't be fixed.

[pitrou]

I'm not sure the race condition possibility is important. Is
shutil.copytree() protected against such race conditions in the first place?

(I'd argue this bug is less about potential DOS attacks than the simple
unfriendliness of indefinitely blocking)

[pitrou]

Here is a patch, introducing a new exception named SpecialFileError
which is raised when trying to copy a named pipe. Other kinds of special
file aren't checked for, but they could it we thought it's sensible to
do so.

[pitrou]

I forgot the case where the /destination/ file is a named pipe. Here is
a new patch.

[pitrou]

Committed in r72178, r72179. Thanks!