Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wrong operator ordering in FST for operators of equal precedence #162

Open
EmilyBourne opened this issue Mar 11, 2020 · 5 comments · May be fixed by #163
Open

Wrong operator ordering in FST for operators of equal precedence #162

EmilyBourne opened this issue Mar 11, 2020 · 5 comments · May be fixed by #163

Comments

@EmilyBourne
Copy link

EmilyBourne commented Mar 11, 2020 •
edited
Loading

When a modulo operator is combined with a multiplication baron does not interpret the code in the same way as python.

E.g:

a % 2 * 4

In python this is equivalent to:

(a % 2) * 4

however baron reads it as:

a % (2 * 4)

as shown here:

>from baron import parse

> fst = parse("a % 2 * 4")

> fst[0]

{'first': {'type': 'name', 'value': 'a'},
 'first_formatting': [{'type': 'space', 'value': ' '}],
 'second': {'first': {'section': 'number', 'type': 'int', 'value': '2'},
  'first_formatting': [{'type': 'space', 'value': ' '}],
  'second': {'section': 'number', 'type': 'int', 'value': '4'},
  'second_formatting': [{'type': 'space', 'value': ' '}],
  'type': 'binary_operator',
  'value': '*'},
 'second_formatting': [{'type': 'space', 'value': ' '}],
 'type': 'binary_operator',
 'value': '%'}
@EmilyBourne EmilyBourne mentioned this issue Mar 11, 2020
Open
@EmilyBourne
Copy link
Author

This is also true for division and multiplication:

a / 2 * 4

gives:

{'first': {'type': 'name', 'value': 'a'},
 'first_formatting': [{'type': 'space', 'value': ' '}],
 'second': {'first': {'section': 'number', 'type': 'int', 'value': '2'},
  'first_formatting': [{'type': 'space', 'value': ' '}],
  'second': {'section': 'number', 'type': 'int', 'value': '4'},
  'second_formatting': [{'type': 'space', 'value': ' '}],
  'type': 'binary_operator',
  'value': '*'},
 'second_formatting': [{'type': 'space', 'value': ' '}],
 'type': 'binary_operator',
 'value': '/'}

@EmilyBourne
Copy link
Author

Operators of equal precedence should be parsed in reverse order

@EmilyBourne EmilyBourne changed the title Wrong priority ordering in FST for modulo operator combined with multiplication Wrong operator ordering in FST for operators of equal precedence Mar 11, 2020
@EmilyBourne
Copy link
Author

EmilyBourne commented Mar 11, 2020 •
edited
Loading

This is happening due to the precedence specified for rply in grammator_operators.py

On lines 191-214:

@pg.production("expr : xor_expr VBAR expr")
    @pg.production("xor_expr : and_expr CIRCUMFLEX xor_expr")
    @pg.production("and_expr : shift_expr AMPER and_expr")
    @pg.production("shift_expr : arith_expr RIGHT_SHIFT shift_expr")
    @pg.production("shift_expr : arith_expr LEFT_SHIFT shift_expr")
    @pg.production("arith_expr : term PLUS arith_expr")
    @pg.production("arith_expr : term MINUS arith_expr")
    @pg.production("term : factor STAR term")
    @pg.production("term : factor SLASH term")
    @pg.production("term : factor PERCENT term")
    @pg.production("term : factor DOUBLE_SLASH term")
    @pg.production("term : factor AT term")
    @pg.production("power : atom DOUBLE_STAR factor")
    @pg.production("power : atom DOUBLE_STAR power")
    def binary_operator_node(pack):
        (first, operator, second) = pack
        return {
            "type": "binary_operator",
            "value": operator.value,
            "first": first,
            "second": second,
            "first_formatting": operator.hidden_tokens_before,
            "second_formatting": operator.hidden_tokens_after
        }

This syntax means that equivalent terms are saved on the right hand side. Thus for the following expression (test_combine_div_modulo_mult in test_grammator_operators.py):

"a/b%c*d"

The operator / is considered first. The term "b%c*d" returns a term so it is saved in the right hand side.

Reversing the syntax to:

    @pg.production("expr : expr VBAR xor_expr")
    @pg.production("xor_expr : xor_expr CIRCUMFLEX and_expr")
    @pg.production("and_expr : and_expr AMPER shift_expr")
    @pg.production("shift_expr : shift_expr RIGHT_SHIFT arith_expr")
    @pg.production("shift_expr : shift_expr LEFT_SHIFT arith_expr")
    @pg.production("arith_expr : arith_expr PLUS term")
    @pg.production("arith_expr : arith_expr MINUS term")
    @pg.production("term : term STAR factor")
    @pg.production("term : term SLASH factor")
    @pg.production("term : term PERCENT factor")
    @pg.production("term : term DOUBLE_SLASH factor")
    @pg.production("term : term AT factor")
    @pg.production("power : factor DOUBLE_STAR atom")
    @pg.production("power : power DOUBLE_STAR atom")
    def binary_operator_node(pack):
        (first, operator, second) = pack
        return {
            "type": "binary_operator",
            "value": operator.value,
            "first": first,
            "second": second,
            "first_formatting": operator.hidden_tokens_before,
            "second_formatting": operator.hidden_tokens_after
        }

leads to the tree being constructed correctly. In this case:

a % 2 * 4

gives:

{'first': {'first': {'type': 'name', 'value': 'a'},
  'first_formatting': [{'type': 'space', 'value': ' '}],
  'second': {'section': 'number', 'type': 'int', 'value': '2'},
  'second_formatting': [{'type': 'space', 'value': ' '}],
  'type': 'binary_operator',
  'value': '%'},
 'first_formatting': [{'type': 'space', 'value': ' '}],
 'second': {'section': 'number', 'type': 'int', 'value': '4'},
 'second_formatting': [{'type': 'space', 'value': ' '}],
 'type': 'binary_operator',
 'value': '*'}

@EmilyBourne
Copy link
Author

The ordering only needs to be changed for non-commutative operators (i.e. all operators but *). This allows the intuitive ordering to be kept for an example such as:

a * b * c

where first is a and second is a*b

@EmilyBourne
Copy link
Author

The ordering only needs to be changed for non-commutative operators (i.e. all operators but *). This allows the intuitive ordering to be kept for an example such as:

a * b * c

where first is a and second is a*b

Unfortunately this is not possible. Objects of equal precedence must be examined in the same direction

@EmilyBourne EmilyBourne linked a pull request Mar 11, 2020 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Issue 162 EmilyBourne/baron
1 participant
@EmilyBourne