re.escape does not correctly escape newlines

[MartinAltmayer]

BPO	42668
Nosy	@ezio-melotti, @MartinAltmayer, @serhiy-storchaka
Files	test.py: simple test file

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-12-17.14:41:45.387>
created_at = <Date 2020-12-17.12:00:52.118>
labels = ['expert-regex', '3.7', '3.8', '3.9', '3.10', 'type-feature', 'invalid']
title = 're.escape does not correctly escape newlines'
updated_at = <Date 2020-12-17.14:41:45.387>
user = 'https://github.com/MartinAltmayer'

bugs.python.org fields:

activity = <Date 2020-12-17.14:41:45.387>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2020-12-17.14:41:45.387>
closer = 'serhiy.storchaka'
components = ['Regular Expressions']
creation = <Date 2020-12-17.12:00:52.118>
creator = 'MartinAltmayer'
dependencies = []
files = ['49689']
hgrepos = []
issue_num = 42668
keywords = []
message_count = 4.0
messages = ['383237', '383239', '383240', '383247']
nosy_count = 4.0
nosy_names = ['ezio.melotti', 'mrabarnett', 'MartinAltmayer', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue42668'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

[MartinAltmayer]

re.escape('\n') returns '\\\n', i.e. a string consisting of a backslash and a newline. I believe it should return '\\n', i.e. a backslash and an 'n'. If the escape-result still contains a verbatim newline, why escape this character at all?

Note that Python's regular expressions engine allows newlines, so re.match(re.escape('\n'), '\n') gives a match. Thus, while this looks like an undesired behavior, it is not functionally broken.

The same problem applies to some other characters: \t\r\v\f

[mrabarnett]

In a regex, putting a backslash before any character that's not an ASCII-range letter or digit makes it a literal. re.escape doesn't special-case control characters. Its purpose is to make a string that might contain metacharacters into one that's a literal, and it already does that, although it sometimes escapes more than necessary.

[serhiy-storchaka]

If the escape-result still contains a verbatim newline, why escape this character at all?

Because in verbose mode it is ignored. This is why \n and other ASCII whitespace characters, and '#', which starts a comment, should be escaped.

'\\\n' and '\\n' has identical meaning independent of mode and context, so there is no bug here. Escaping '\n' as '\\\n' is slightly simpler.