A lexical analyzer in JavaScript.
JavaScript
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
jslex
licenses
test
README.md
package.json

README.md

jslex

Lexical analyzer JavaScript library. Similar to lex and flex.

Developed by

  • Jim R. Wilson (jimbojw)

GitHub repository

License

This project is released under The MIT License.

Usage

The best way to understand how to use jslex is via illustrative example.

First, create a lexer:

var lexer = new jslex( {
    "start": {
        "[0-9]+": function() {
            return parseInt(this.text, 10);
        },
        "[-+\n]": function() {
            return this.text;
        },
        "[ \t\r]": null,
        ".": function() {
            throw "Invalid character '" + this.text + "' (line:" + (1 + this.line) + ", column:" + this.column + ")";
        }
    }
} );

The above lexer has one state, called "start", which contains 4 patterns:

  • one for numbers,
  • one for plus, minus, or new lines,
  • one for uninteresting whitespace characters, and
  • one for anything else.

To use the lexer on an input string, you can use the lex() method:

function callback( token ) {
  // Do something with returned token
}

lexer.lex( "1 + 2", callback );

In the above example, the callback() function will be called three times:

  • first with the number 1,
  • then with the string "+", and
  • lastly with the number 2.

A very common use case for the lexer is to simply collect all the tokens in an input stream. For this, use the collect() function:

var tokens = lexer.collect( "1 + 2" );

After executing the above snippet, the tokens variable would have three elements: [1, "+", 2].