Skip to content

Commit

Permalink
Fix an AST and token incompatibility for Prism::Translation::Parser
Browse files Browse the repository at this point in the history
Fixes #2515.

This PR fixes an AST and token incompatibility between Parser gem and `Prism::Translation::Parser`
for string literal with line breaks.
  • Loading branch information
koic committed Mar 12, 2024
1 parent c5f5a02 commit c58466e
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 2 deletions.
16 changes: 15 additions & 1 deletion lib/prism/translation/parser/compiler.rb
Expand Up @@ -1487,9 +1487,23 @@ def visit_string_node(node)
elsif node.opening == "?"
builder.character([node.unescaped, srange(node.location)])
else
parts = if node.unescaped.lines.count <= 1
[builder.string_internal([node.unescaped, srange(node.content_loc)])]
else
start_offset = node.content_loc.start_offset

node.unescaped.lines.map do |line|
end_offset = start_offset + line.length
offsets = srange_offsets(start_offset, end_offset)
start_offset = end_offset

builder.string_internal([line, offsets])
end
end

builder.string_compose(
token(node.opening_loc),
[builder.string_internal([node.unescaped, srange(node.content_loc)])],
parts,
token(node.closing_loc)
)
end
Expand Down
2 changes: 1 addition & 1 deletion lib/prism/translation/parser/lexer.rb
Expand Up @@ -281,7 +281,7 @@ def to_a
value = ""
location = Range.new(source_buffer, offset_cache[next_location.start_offset], offset_cache[next_location.end_offset])
index += 1
elsif ["\"", "'"].include?(value) && (next_token = lexed[index][0]) && next_token.type == :STRING_CONTENT && (next_next_token = lexed[index + 1][0]) && next_next_token.type == :STRING_END
elsif ["\"", "'"].include?(value) && (next_token = lexed[index][0]) && next_token.type == :STRING_CONTENT && next_token.value.lines.count <= 1 && (next_next_token = lexed[index + 1][0]) && next_next_token.type == :STRING_END
next_location = token.location.join(next_next_token.location)
type = :tSTRING
value = next_token.value
Expand Down
2 changes: 2 additions & 0 deletions test/prism/fixtures/dstring.txt
@@ -0,0 +1,2 @@
"foo
bar"
11 changes: 11 additions & 0 deletions test/prism/snapshots/dstring.txt

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit c58466e

Please sign in to comment.