Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix a token incompatibility for
Prism::Translation::Parser::Lexer
This PR fixes a token incompatibility between Parser gem and `Prism::Translation::Parser` for beginless range: ## Parser gem (Expected) Returns `tBDOT2` token: ```console $ bundle exec ruby -Ilib -rparser/ruby33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Parser::Ruby33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] [[:tBDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]] ``` ## `Prism::Translation::Parser` (Actual) Previously, the parser returned `tDOT2` token when parsing the following: ```console $ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Prism::Translation::Parser33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] [[:tDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]] ``` After the update, the parser now returns `tBDOT2` token for the same input: ```console $ bundle exec ruby -Ilib -rprism -rprism/translation/parser33 -ve \ 'buf = Parser::Source::Buffer.new("example.rb"); buf.source = "..42"; p Prism::Translation::Parser33.new.tokenize(buf)[2]' ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] [[:tBDOT2, ["..", #<Parser::Source::Range example.rb 0...2>]], [:tINTEGER, [42, #<Parser::Source::Range example.rb 2...4>]]] ``` This correction enables the restoration of `endless_range_in_conditional.txt` as a test case.
- Loading branch information