Skip to content

PyREPL doesn't correctly color double braces in f-strings / t-strings #134158

@loic-simon

Description

@loic-simon

Bug report

Bug description:

Syntax highlighting in PyREPL (added in #131507) does not correctly color the double-braces used for escaping braces in f-strings or t-strings:

Image

After investigation, this is caused by Python tokenizer producing only 1 brace in this case:

>>> from tokenize import tokenize
>>> from io import BytesIO
>>> from pprint import pp
>>> 
>>> pp(list(tokenize(BytesIO(b'f"a{{b"').readline)))
[TokenInfo(type=68 (ENCODING), string='utf-8', start=(0, 0), end=(0, 0), line=''),
TokenInfo(type=59 (FSTRING_START), string='f"', start=(1, 0), end=(1, 2), line='f"a{{b"'),
TokenInfo(type=60 (FSTRING_MIDDLE), string='a{', start=(1, 2), end=(1, 4), line='f"a{{b"'),
TokenInfo(type=60 (FSTRING_MIDDLE), string='b', start=(1, 5), end=(1, 6), line='f"a{{b"'),
TokenInfo(type=61 (FSTRING_END), string='"', start=(1, 6), end=(1, 7), line='f"a{{b"'),
TokenInfo(type=4 (NEWLINE), string='', start=(1, 7), end=(1, 8), line='f"a{{b"'),
TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]

So PyREPL think there is only one brace in the original string, and doesn't color the second one.

I have a fix for this issue, PR on the way!

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

Windows, macOS

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.14bugs and security fixes3.15new features, bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions