Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Unput leaves yytext one character short #135

Closed
wants to merge 2 commits into from

3 participants

@petkaantonov

There is an off-by-one error in the unput method that leaves the yytext shorter. This should fix it.

@zaach
Owner

Awesome. Could you submit this PR to jison-lex instead? The lexer has been moved there, to be used as a dependency for the next version of jison. The lexer and test files should basically be the same.

@zaach
Owner

Fixed in latest jison-lex.

@zaach zaach closed this
@fgnass

I just ran into the same issue. If I'm not mistaken jison-lex still contains the buggy line:
https://github.com/zaach/jison-lex/blob/master/regexp-lexer.js#L209

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 29, 2013
  1. @petkaantonov
  2. @petkaantonov

    Fix off-by-one error

    petkaantonov authored
This page is out of date. Refresh to see the latest.
Showing with 21 additions and 1 deletion.
  1. +1 −1  lib/jison/lexer.js
  2. +20 −0 tests/lexer/regexplexer.js
View
2  lib/jison/lexer.js
@@ -184,7 +184,7 @@ RegExpLexer.prototype = {
var lines = ch.split(/(?:\r\n?|\n)/g);
this._input = ch + this._input;
- this.yytext = this.yytext.substr(0, this.yytext.length-len-1);
+ this.yytext = this.yytext.substr(0, this.yytext.length-len);
//this.yyleng -= len;
this.offset -= len;
var oldLines = this.match.split(/(?:\r\n?|\n)/g);
View
20 tests/lexer/regexplexer.js
@@ -893,3 +893,23 @@ exports["test unput location again"] = function() {
};
+exports["test yytext state after unput"] = function() {
+ var dict = {
+ rules: [
+ ["cat4", "this.unput('4'); return 'CAT';" ],
+ ["4", "return 'NUMBER';" ],
+ ["$", "return 'EOF';"]
+ ]
+ };
+
+ var input = "cat4";
+
+ var lexer = new RegExpLexer(dict);
+ lexer.setInput(input);
+ assert.equal(lexer.lex(), "CAT");
+ /*the yytext should be 'cat' since we unput '4' from 'cat4' */
+ assert.equal(lexer.yytext, "cat");
+ assert.equal(lexer.lex(), "NUMBER");
+ assert.equal(lexer.lex(), "EOF");
+};
+
Something went wrong with that request. Please try again.