Docs?

[matthewkastor]

I wanted to try out a few things today with jison. I was looking through the code and it wasn't immediately obvious how to call jison-lex from scripts. While working out the lex section of my jison file I thought it would be good if I could just use jison-lex directly from a script, that way I could write a test in jasmine-node and do something like jasmine-node spec/ --autotest while I worked out what I was doing.

Here's something I wrote after reading through the source code. I'm posting it here in case you might want to use it in the readme.

var fs = require('fs');
var jisonLex = require('jison-lex');

var grammar = fs.readFileSync('./comments.jison-lex', 'utf8');
var input = fs.readFileSync('./comments.txt', 'utf8');

var lexer = new jisonLex(grammar, input);

function saveLexerToFile () {
    var mymodule = lexer.generate({
        moduleType : 'commonjs',
        moduleName : 'comments'
    });
    fs.writeFileSync("./comments.lex.js", mymodule);
}

// grabs tokens from whatever lexer is available.
function getResults () {
    var result = lexer.lex();
    console.log(result);
    if(result !== 'EOF') {
        setTimeout(getResults);
    }
}


// replaces lexer instance with the lexer stored in the file.
function runLexerFile () {
    lexer = require("./comments.lex.js").lexer;
    lexer.setInput(input);
    getResults();
}


getResults();

[Ravenwater]

This was the missing example for me:

var fs  = require('fs')
var JisonLex = require('jison-lex');

var syntax = fs.readFileSync('mylexfile.l', 'utf8');
var lexer  = new JisonLex(syntax);
var input  = fs.readFileSync('mylanguagefile', 'utf8');

lexer.setInput(input);
var token = lexer.lex();
while (token !== 'ENDOFFILE') {      // <-- of course this implies this token in your lex specification
    console.log('Token : ' + token);
    token = lexer.lex();
}
console.log('Done...');

This will tokenize the mylanguagefile according to the syntax defined in mylexfile.l. Now I have a minimum command line utility to experiment with the syntax and/or language specification.