Document the bug in re.findall() and re.finditer() in 2.7 and 3.6

[serhiy-storchaka]

BPO	32211
Nosy	@rhettinger, @ezio-melotti, @serhiy-storchaka
PRs	[3.6] bpo-32211: Document the existing bug in re.findall() and re.finditer(). #4695 [2.7] bpo-32211: Document the existing bug in re.findall() and re.finditer(). (GH-4695). #5096

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-01-04.09:34:08.592>
created_at = <Date 2017-12-04.10:08:06.272>
labels = ['expert-regex', 'type-feature', 'docs']
title = 'Document the bug in re.findall() and re.finditer() in 2.7 and 3.6'
updated_at = <Date 2018-01-04.12:08:33.773>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2018-01-04.12:08:33.773>
actor = 'serhiy.storchaka'
assignee = 'docs@python'
closed = True
closed_date = <Date 2018-01-04.09:34:08.592>
closer = 'serhiy.storchaka'
components = ['Documentation', 'Regular Expressions']
creation = <Date 2017-12-04.10:08:06.272>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 32211
keywords = ['patch']
message_count = 3.0
messages = ['307546', '309459', '309463']
nosy_count = 5.0
nosy_names = ['rhettinger', 'ezio.melotti', 'mrabarnett', 'docs@python', 'serhiy.storchaka']
pr_nums = ['4695', '5096']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue32211'
versions = ['Python 2.7', 'Python 3.6']

[serhiy-storchaka]

>>> re.findall(r'^|\w+', 'two words')
['', 'wo', 'words']

Seems the current behavior was documented incorrectly in bpo-732120.

It will be fixed in 3.7 (see bpo-1647489, bpo-25054), but I hesitate to backport the fix to 3.6 and 2.7 because this can break the user code. For example:

In Python 3.6:

>>> list(re.finditer(r'(?m)^\s*?$', 'foo\n\n\nbar'))
[<_sre.SRE_Match object; span=(4, 4), match=''>, <_sre.SRE_Match object; span=(5, 5), match=''>]

In Python 3.7:

>>> list(re.finditer(r'(?m)^\s*?$', 'foo\n\n\nbar'))
[<re.Match object; span=(4, 4), match=''>, <re.Match object; span=(4, 5), match='\n'>, <re.Match object; span=(5, 5), match=''>]

(This is a real pattern used in the docstring module, but with re.sub()).

The proposed PR documents the current weird behavior in 2.7 and 3.6.

[serhiy-storchaka]

New changeset 1e6d852 by Serhiy Storchaka in branch '3.6':
bpo-32211: Document the existing bug in re.findall() and re.finditer(). (bpo-4695)
1e6d852

[serhiy-storchaka]

New changeset ca54740 by Serhiy Storchaka in branch '2.7':
[2.7] bpo-32211: Document the existing bug in re.findall() and re.finditer(). (GH-4695). (bpo-5096)
ca54740