(PUP-9163) Fix lexer problem with heredoc and sublocator #7114
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jhelwig
reviewed
Sep 24, 2018
| sublexer = self.class.new() | ||
| locator = Locator::SubLocator.sub_locator(str, | ||
| locator.file, heredoc_line, heredoc_offset, leading.length()) | ||
| locator = Locator::SubLocator.new(locator, heredoc_line, heredoc_offset, leading.length()) |
There was a problem hiding this comment.
meh - fixed - new version force pushed
Before this, heredoc set up a locator intended to define a subspace for the heredoc string (for lexing its content to support interpolation). What it ended up doing was to create a subspace offset not from the total source string, but from within the heredoc string. This resulted in offsets being calculated to point to char positions in an undefined manner (depending of length of heredoc, its position and what follows). This would then result in error messages pointing to the wrong place in the source code as well as wrongly computing if a `[` is an array start or not by checking if the preceding char is a space since it would end up looking at some undefined position. This was made worse (or the same) because the lexer would use the returned char_offset as an offset in its subspace string instead of in the total space.
hlindberg
force-pushed
the
PUP-9163_locator-for-heredoc-subspace-is-wrong
branch
from
fe0ad58 to
49b42a5
Compare
jhelwig
approved these changes
Sep 24, 2018
|
CLA signed by all contributors. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before this, heredoc set up a locator intended to define a subspace
for the heredoc string (for lexing its content to support
interpolation). What it ended up doing was to create a subspace offset
not from the total source string, but from within the heredoc string.
This resulted in offsets being calculated to point to char positions in
an undefined manner (depending of length of heredoc, its position and
what follows). This would then result in error messages pointing to the
wrong place in the source code as well as wrongly computing if a
[isan array start or not by checking if the preceding char is a space since
it would end up looking at some undefined position.
This was made worse (or the same) because the lexer would use the
returned char_offset as an offset in its subspace string instead of in
the total space.