preparser incorrectly handles backslash operator inside strings (sometimes)

[sagetrac-cwitty]

When reviewing #5595, I tried typing this:

sage: import re
sage: dep_regex = re.compile(r'^ *(?:(?:cimport +([\w\. ,]+))|(?:from +(\w+) +cimport)|(?:include *[\'"]([^\'"]+)[\'"])|(?:cdef *extern *from *[\'"]([^\'"]+)[\'"]))', re.M)
------------------------------------------------------------
   File "<ipython console>", line 1
     dep_regex = re.compile(r'^ *(?:(?:cimport +([\w\. ,]+))|(?:from +(\w+) +cimport)|(?:include *[\'"]([^\'"]+)[ * BackslashOperator() * '"])|(?:cdef *extern *from *[\'"]([^\'"]+)[\'"]))', re.M)
                                                                                                                                                                          ^
SyntaxError: invalid syntax

Note that the preparser has turned a backslash from the middle of the regular expression into " * BackslashOperator() * ".

CC: @robertwb

Component: user interface

Keywords: preparser

Issue created by migration from https://trac.sagemath.org/ticket/5821

[robertwb]

comment:1

It appears that even in raw strings the backslash escapes a potential end quote...

http://docs.python.org/reference/lexical_analysis.html

[robertwb]

comment:2

Attachment: 5821-preparse-raw.patch.gz

[sagetrac-cwitty]

comment:3

Code looks good (and fixes the problem); doctests pass.

Positive review.

[sagetrac-mabshoff]

comment:4

Merged in Sage 3.4.2.alpha0.

Cheers,

Michael