raw strings cannot end with a backslash character r'\'

[gpshead]

BPO	31136
Nosy	@tim-one, @gpshead, @vstinner, @ericvsmith, @bitdancer, @vadmium

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-08-08.02:28:28.256>
created_at = <Date 2017-08-08.00:50:49.474>
labels = ['interpreter-core', '3.7', 'invalid']
title = "raw strings cannot end with a backslash character r'\\'"
updated_at = <Date 2019-02-21.02:29:54.398>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2019-02-21.02:29:54.398>
actor = 'gwideman'
assignee = 'none'
closed = True
closed_date = <Date 2017-08-08.02:28:28.256>
closer = 'tim.peters'
components = ['Interpreter Core']
creation = <Date 2017-08-08.00:50:49.474>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 31136
keywords = []
message_count = 9.0
messages = ['299883', '299885', '299889', '299890', '299893', '299918', '299934', '336174', '336175']
nosy_count = 7.0
nosy_names = ['tim.peters', 'gregory.p.smith', 'vstinner', 'eric.smith', 'r.david.murray', 'gwideman', 'martin.panter']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue31136'
versions = ['Python 3.7']

[gpshead]

A raw string literal cannot end in a backslash. There is no friendly way to write a string that ends in a backslash character.

In particular I want to put the following into a Python string: \\?\

'\\\\?\\' works but is escaping hell so I wanted to suggest to the author to use r'\\?\' but that leads to:
SyntaxError: EOL while scanning string literal

Tested in a random 3.7.0a0 build as well as older 2.7 and 3.x stable versions.

r'\' is the easiest way to reproduce this. (which could be written using the same number of bytes as '\\'... the use case above where a string containing a lot of \s that also ends in a \ is where it matters more from a code beauty point of view)

Can we update the parser to allow this?

[vstinner]

Workaround working on all Python versions, string concatenation made by the compiler: r"...." "...".

>>> print(r"a\n" "\\")
a\n\

[vadmium]

What would your proposal do where an embedded backslash is currently valid?

>>> print(r'Backslash apostrophe: \'.')
Backslash apostrophe: \'.
>>> r'\'  # Comment or string?'
"\\'  # Comment or string?"

[gpshead]

This may well be a "not a bug" resolution to preserve existing semantics that weren't what I expected.

[tim-one]

Yes, I'm closing as not-a-bug. It's been this way (and documented) forever. More specifically, as the docs say, a raw string can't end with an _odd_ number of backslashes:

"""
String quotes can be escaped with a backslash, but the backslash remains in the string; for example, r"\"" is a valid string literal consisting of two characters: a backslash and a double quote; r"\" is not a valid string literal (even a raw string cannot end in an odd number of backslashes). Specifically, a raw string cannot end in a single backslash (since the backslash would escape the following quote character).
"""

[bitdancer]

In fact, this ia a FAQ:

https://docs.python.org/3/faq/design.html#why-can-t-raw-strings-r-strings-end-with-a-backslash

[gpshead]

If i could stop thinking of r as meaning "raw" as we document it and instead "regular expression literal" I wouldn't make this mistake. :)

Thanks everyone!

[gwideman]

Let us be clear here that this is NOT a case where the backslash escapes the subsequent quote. If it WAS such a case, then the sequence \' would leave only the quote in the output string. But it doesn't; it leaves the complete 2-character \' in the output string.
So essentially this is a case of the character sequence \' being given a special status that causes that character pair to have a special meaning in preference to the meaning of the individual characters.
So this IS a bug -- it may be "as designed", but that produces the bug in the name of this feature, "raw string", which is patently misleading and in violation of the principle of least surprise. This is a feature (as the FAQ explains) provided explicitly for developers of regular expression parsers. So at best, these r-strings should be called "regex-oriented" string literals, which can be used elsewhere, at risk of knowing this gotcha.

[gwideman]

Demonstration:
print("x" + r' \' ' + "x") produces
x \' x
Where is this behavior _ever_ useful?
Or if there is some use case for this, how frequent is it compared to the frequency of users expecting either that backslash does nothing special, or that it would behave like an escape, and not appear in the output?

I'm not here to suggest there's some easy fix for this. I just don't want this issue closing as "not a bug" and fail to register that this design is flawed.