Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect parsing of Unicode Literals. Got - UntreatedError: Untreated elements #208

Open
Erotemic opened this issue Oct 27, 2021 · 0 comments

Comments

@Erotemic
Copy link

Erotemic commented Oct 27, 2021

The following code illustrates an issue in redbaron 0.9.2 where it fails to parse unicode literals

    import textwrap
    import redbaron

    # This works properly
    text = textwrap.dedent(
        '''
        p1, p2 = (1, 2)
        ''').strip('\n')
    red = redbaron.RedBaron(text)

    # But this fails when we use unicode symbols for identifiers
    text = textwrap.dedent(
        '''
        ρ1, ρ2 = (1, 2)
        ''').strip('\n')
    red = redbaron.RedBaron(text)

    # Still fails with a single unicdoe element
    text = textwrap.dedent(
        '''
        ρ2 = 2
        ''').strip('\n')
    red = redbaron.RedBaron(text)

    # Still fails with different unicode identifiers even with explicit
    # unicode literal futures
    text = textwrap.dedent(
        '''
        from __future__ import unicode_literals
        θ = 2
        ''').strip('\n')
    red = redbaron.RedBaron(text)

Essentially, using a unicode character should be valid for a variable name in Python 3, but redbaron does not seem to play nicely with that case.

    # System information
    import sys
    print('sys.version_info = {!r}'.format(sys.version_info))
    import ubelt as ub
    _ = ub.cmd('pip list | grep redbaron', shell=True, verbose=1)

Results in:

sys.version_info = sys.version_info(major=3, minor=8, micro=6, releaselevel='final', serial=0)
redbaron                          0.9.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant