glob.glob handling of * (asterisk) wildcard is broken

[max0x7ba]

BPO	44380
Nosy	@akheron, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-06-10.15:53:43.363>
labels = ['3.11', 'type-feature', '3.9', '3.10', 'docs']
title = 'glob.glob handling of * (asterisk) wildcard is broken'
updated_at = <Date 2021-06-10.16:33:19.945>
user = 'https://bugs.python.org/max0x7ba'

bugs.python.org fields:

activity = <Date 2021-06-10.16:33:19.945>
actor = 'max0x7ba'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2021-06-10.15:53:43.363>
creator = 'max0x7ba'
dependencies = []
files = []
hgrepos = []
issue_num = 44380
keywords = []
message_count = 5.0
messages = ['395545', '395547', '395548', '395550', '395551']
nosy_count = 6.0
nosy_names = ['docs@python', 'nailor', 'petri.lehtinen', 'max0x7ba', 'serhiy.storchaka', 'Isaac Muse']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue44380'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[max0x7ba]

Problem:

glob.glob documentation states that "pathname ... can contain shell-style wildcards."

However, it stops short of saying that shell-style wildcards are handled the same way as in a POSIX-compliant/friendly shell.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13_02 POSIX requires that "* (asterisk) is a pattern that shall match any string, including the null string."

However, glob.glob pattern * (asterisk) doesn't match an empty/null string.

Reproduction:

$ ls *.bash_profile
.bash_profile
$ python3 -c 'import glob; print(glob.glob("*.bash_profile"))'
[]
$ python3 -c 'import glob; print(glob.glob(".bash_profile"))'
['.bash_profile']

[serhiy-storchaka]

Only pattern beginning with a dot can match filename beginning with a dot.

From https://docs.python.org/3/library/glob.html

Note that unlike fnmatch.fnmatch(), glob treats filenames beginning with a dot (.) as special cases.

This phrase was added in bpo-16695 in attempt to improve documentation, but it is still not clear.

[max0x7ba]

I may be naive, but why then:

$ python3 -c 'from pathlib import Path; print(list(Path(".").glob("*.bash_profile")))'

Outputs:

[PosixPath('.bash_profile')]

?

[IsaacMuse]

Sadly, this because pathlib glob and glob.glob use different implementations. And glob.glob does not provide something equivalent to a DOTALL flag allowing a user to glob hidden files without explicitly defining the leading dot in the pattern.

[max0x7ba]

glob.glob does not provide something equivalent to a DOTALL flag

I see now, said a blind man.

[hauntsaninja]

While a little surprising, this has been the behaviour of glob since 1991 (the relevant check looked like if name[0] <> '.' or pattern[0] = '.': at the time). It's been explicitly documented for a decade, and now thanks to this issue, more clearly documented.