Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update string nodes for implicit concatenation #7927

Merged
merged 14 commits into from
Nov 24, 2023

Commits on Nov 22, 2023

  1. Update existing string nodes for implicit concatenation

    This commit implements the AST design to account for implicit
    concatenation in string nodes, specifically the `ExprFString`,
    `ExprStringLiteral`, and `ExprBytesLiteral` nodes.
    dhruvmanila committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    86d799d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    223380f View commit details
    Browse the repository at this point in the history
  3. Update parser snapshots

    dhruvmanila committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    21ed36c View commit details
    Browse the repository at this point in the history
  4. Add AnyNode/AnyNodeRef variants for string parts

    This commit adds the new variants for the string parts to `AnyNode` and
    `AnyNodeRef` enums. These parts are `StringLiteral` (part of
    `ExprStringLiteral`), `BytesLiteral` (part of `ExprBytesLiteral`), and
    `FString` (part of `ExprFString`).
    
    The reason for this is to add visitor methods for these parts. This is
    done in the following commit. So, the visitor would visit the string as
    a whole first and then visit each part.
    
    ```
    ExprStringLiteral - "foo" "bar"
    |- StringLiteral - "foo"
    |- StringLiteral - "bar"
    ```
    
    The above tree helps understand the way visitor would work.
    dhruvmanila committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    0c6e2e7 View commit details
    Browse the repository at this point in the history
  5. Add methods for string parts in visitor implementation

    The visitor implementations are updated to visit each part nodes for the
    respective string nodes.
    
    The following example better highlights this:
    ```
    ExprStringLiteral - "foo" "bar"
    |- StringLiteral - "foo"
    |- StringLiteral - "bar"
    ```
    
    The `visit_expr` method would be use to visit the `ExprStringLiteral`
    while the `visit_string_literal` method would be use for the
    `StringLiteral` node. Similar methods are added for bytes and f-strings.
    dhruvmanila committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    59ef011 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    6be8eca View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    1c6dc52 View commit details
    Browse the repository at this point in the history
  8. Update the Generator for the new AST design

    The generator is basically improved. Earlier, for an implicitly
    concatenated string we would produce the joined form. So,
    
    ```python
    "foo" "bar" "baz"
    ```
    
    For the above example, the generator would give us:
    ```python
    "foobarbaz"
    ```
    
    Now, as we have the information for each part, we will be producing the
    exact code back.
    dhruvmanila committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    da89c76 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    6cdf149 View commit details
    Browse the repository at this point in the history
  10. Move is_implicit_concatenated to a narrow type

    `Expr` is a general type for all expressions while
    `LiteralExpressionRef` is a type which includes only the literal
    expressions. The method is suited more for this type instead.
    
    This will also help in the formatter change.
    dhruvmanila committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    95f71da View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    6d6e1c7 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    bcabcca View commit details
    Browse the repository at this point in the history
  13. Implement PartialEq for ConcatenatedStringLiteral

    As highlighted in the review:
    
    > If you have two `ConcatenatedStringLiteral` values where both have
    > equivalent values for `strings` but where one has `value` initialized
    > and the other does not, would you expect them to compare equal?
    > Semantically I think I would, since the alternative is that equality is
    > dependent on whether `as_str()` has been called, which seems incidental.
    
    #7927 (comment)
    dhruvmanila committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    8193ff9 View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2023

  1. Configuration menu
    Copy the full SHA
    1e72af2 View commit details
    Browse the repository at this point in the history