os.listdir is missing in os.supports_dir_fd

[hynek]

BPO	15217
Nosy	@pitrou, @larryhastings, @hynek, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/larryhastings'
closed_at = <Date 2012-06-28.23:50:37.054>
created_at = <Date 2012-06-28.12:04:20.275>
labels = ['invalid', 'type-bug', 'library', 'release-blocker']
title = 'os.listdir is missing in os.supports_dir_fd'
updated_at = <Date 2012-06-29.08:24:25.983>
user = 'https://github.com/hynek'

bugs.python.org fields:

activity = <Date 2012-06-29.08:24:25.983>
actor = 'larry'
assignee = 'larry'
closed = True
closed_date = <Date 2012-06-28.23:50:37.054>
closer = 'larry'
components = ['Library (Lib)']
creation = <Date 2012-06-28.12:04:20.275>
creator = 'hynek'
dependencies = []
files = []
hgrepos = []
issue_num = 15217
keywords = []
message_count = 11.0
messages = ['164249', '164290', '164291', '164292', '164294', '164301', '164314', '164315', '164317', '164318', '164319']
nosy_count = 5.0
nosy_names = ['pitrou', 'larry', 'Arfrever', 'hynek', 'serhiy.storchaka']
pr_nums = []
priority = 'release blocker'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15217'
versions = ['Python 3.3']

[hynek]

>>> n = os.open('/tmp', os.O_RDONLY)
>>> l = os.listdir(n)
>>> os.listdir in os.supports_dir_fd
False

[larryhastings]

I want to mark this as wontfix.

>>> os.listdir in os.supports_fd
True

The concept we're struggling with here: is the "fd" you pass in to os.listdir a "dir_fd"? I claim that it isn't. I'm trying to enforce the concept, in both the documentation and the code, that "dir_fd" is a directory fd *accompanying a path*. With listdir, it isn't accompanied by a path, it replaces the path. So I suggest it's not a "dir_fd", it's just an "fd" which by sheer coincidence happens to reference a directory.

I proposed making os.listdir accept dir_fd, and internally use open and listdir to emulate the behavior. But someone (Antoine?) rightly pointed out, this would break the guideline that POSIX os.* functions on Unix-y OSes are atomic.

I might go so far as to say this needs a documentation fix. But I won't condone any sort of code fix (like adding os.listdir to os.supports_dir_fd).

[hynek]

I don't really care about the name but I'd really like some way to check whether listdir supports fds.

[larryhastings]

Like I said: use os.supports_fd.

[hynek]

Ah sorry, that wasn't in the mail (stupid >>>).

[larryhastings]

I'm closing this. os.listdir should not be listed in supports_dir_fd, because it has no dir_fd parameter.

For reference, we discussed all this previously in issue bpo-15176.

[serhiy-storchaka]

I proposed making os.listdir accept dir_fd, and internally use open and listdir to emulate the behavior.

Hmm, I understood the issue as the request for adding dir_fd argument to
os.listdir. openat+fdopendir+readdir+closedir+close or, in Python,

def listdir2(path, dir_fd=None):
    fd = os.open(path, dir_fd)
    try:
        return os.listdir(fd)
    finally:
        os.close(fd)

But someone (Antoine?) rightly pointed out, this would break the guideline that POSIX os.* functions on Unix-y OSes are atomic.

os.listdir is not atomic. opendir+readdir+closedir+close or fdup
+fdopendir+readdir+closedir+close.

[larryhastings]

Storchaka: please take it up with Antoine, he's the defender of the realm for POSIX-functions-are-atomic iirc. I'd be happy with dir_fd for os.listdir, though it may be too late for 3.3 anyway.

[hynek]

Just in my defense: Larry suggested in msg164245 to me to check for listdir in supports_dir_fd so I just assumed it's broken as it didn't work. I'm totally fine with the current behavior, the discussion about dir_fd support in listdir isn't what I intended with this ticket.

[serhiy-storchaka]

Storchaka: please take it up with Antoine, he's the defender of the realm for POSIX-functions-are-atomic iirc. I'd be happy with dir_fd for os.listdir, though it may be too late for 3.3 anyway.

I know that this is possible (and relatively easy), but I don't think
it's useful. If you use at-functions, then after listdir should be
at-function, and for this purpose you need to get directory's fd in any
case. I don't see useful use cases for listdir(name, dir_fd). So I'm not
going to open a new issue and I'm happy with closing this one.

[larryhastings]

WFM. That is, unless people want to bring up additional only-tangentially-related topics to annoy Hynek ;-)

How about those <local sports team>? They're really having a year of it, aren't they!