parse

Running

Operations that run a parser against a state to and extract the success or failure result.

parse.run(p, input, [userData])

Run parser p against array like object input. An optional [user data][user-data] object can be provided.

run returns success results and throws error results.

parse.run(
    parse_text.string('ab'),
    'abc'); // returns 'ab'

parse.run(
    parse_text.string('ab'),
    'x'); // throws Expected 'a' found 'x'

parse.runStream(p, s, [userData])

Run parser p against a potentially infinite [Nu][nu] stream s. An optional [user data][user-data] object can be provided.

Returns success results, throws error results.

parse.runStream(
    parse_text.string('ab'),
    stream.from('abc')); // returns 'ab'

// Run against infinite, lazy stream
parse.runStream(
    parse_text.string('aaaaa'),
    gen.repeat(Infinity, 'a')); // returns 'aaaaa'

parse.runState(p, state)

Run parser p with state. This allows you to use a custom [parser state][parser-state] or use a [custom position][position].

Returns success results, throws error results.

parse.runState(
    parse_text.string('ab'),
    new parse.ParserState(
         parse.Position.initial,
         stream.from('abc'),
         {});
 // returns 'ab'

parse.test(p, input, [userData])

Same as parse.run but returns a boolean that indicates if parser p succeeded or failed.

parse.test(
    parse_text.string('ab'),
    'abc'); // returns true

parse.run(
    parse_text.string('ab'),
    'x'); // returns false

parse.testStream(p, s, [userData])

Same as parse.runStream but returns a boolean that indicates if parser p succeeded or failed.

parse.testState(p, state)

Same as parse.runState but returns a boolean that indicates if parser p succeeded or failed.

parse.parse(p, input, userData, ok, err)

Runs parser p against an array like input. A [user data][user-data] object must be provided, but may be null. Calls ok if the parser succeeds and err if it fails.

Returns result callback.

parse.parse(
    parse_text.string('ab'),
    'abc',
    null
    \x -> console.log("suc" + x),
    \x -> console.log("error" + x));

parse.parseStream(p, s, userData, ok, err)

Same as parse.parse but takes and [Nu][nu] stream s as input.

parse.parseState(p, state, ok, err)

Same as parse.parse but takes an explicit parser state state.

Basic Parsers

parse.always(x), parse.of(x)

Parser that always succeeds with value x.

var p = parse.always('3');

parse.run(p, ''); // 3
parse.run(p, 'abc'); // 3

parse.never(x)

Parser that always fails with value x.

parse.run(parse.never(5), 'abc'); // throws 5
parse.run(parse.never('Error'), 'abc'); // throws 'Error'

parse.bind(p, f), parse.chain(p, f)

Parser that parses p, passing the result to function f which returns a parser to continue the computation.

var p = parse.bind(parse.always(1), \x -> parse.always(x + 1));
parse.run(p, ''); // 2
    
var err = parse.bind(parse.never(), \x -> parse.always(x + 1));
parse.run(p, ''); // throws UnknownError, 'f' never called.

parse.fail([message])

Parser that always fails with an error.

If message is not provided, fails with UnknownError. Otherwise, fails with 'ParseError' using message

parse.run(parse.never(), 'abc'); // throws UnknownError
parse.run(parse.never('Error'), 'abc'); // throws ParserError with error message "Error"

Sequencing

parse.next(p, q), parse.concat(p, q)

Consumes p then q. Returns result from q

var p = parse.next(
    parse_text.character('a'),
    parse_text.character('b'));

parse.run(p, 'ab'); // b 
parse.run(p, 'abc'); // b
parse.run(p, 'a'); // Error!
parse.run(p, 'aab'); // Error!
parse.run(p, 'ba'); // Error!

// If `p` fails, `q` is never run.
var p = parse.next(
    parse.never(),
    parse_text.character('b'));

parse.run(p, 'b'); // Error
parse.run(p, ''); // Error

parse.sequence(...parsers)

Parser that runs each parser from parsers in order, returning the last result.

var p = parse.sequence(
    parse_text.character('a'),
    parse_text.character('b'),
    parse_text.character('c'));

parse.run(p, 'a'); // Error, expected b
parse.run(p, 'ab'); // Error expected c
parse.run(p, 'abc'); // c
parse.run(p, 'abcd'); // c

parse.sequencea(parsers)

Same as parse.sequence but takes parsers as an array.

var p = parse.sequencea(['a', 'b', 'c'].map(parse_text.character));

parse.run(p, 'a'); // Error, expected b
parse.run(p, 'ab'); // Error expected c
parse.run(p, 'abc'); // c

parse.sequences(parsers)

Same as parse.sequence but get parsers as a [Nu][nu] stream. This allows parsing infinite lazy sequences of parsers.

var tenA = parse.sequences(
    stream.repeat(10, parse_text.character('a'));

parse.run(tenA, 'aaaaaaaaaaaa'); // a

Choice

parse.either(p, q)

Parser that succeeds with either parser p or q. Attempts p first and if p fails attempts q.

If both fail, fails with a MultipleError with errors from p and q combined.

var p = parse.either(
    parse_text.character('a'),
    parse_text.character('b'));

parse.run(p, 'a'); // a 
parse.run(p, 'b'); // b
parse.run(p, 'c'); // Error! MultipleError

Either does not automatically backtrack, use parse.attempt to add backtracking

var p = parse.either(
    parse.next(
        parse_text.character('a'),
        parse_text.character('b')),
    parse.next(
        parse_text.character('a'),
        parse_text.character('c')));
parse.run(p, 'ab'); // b
parse.run(p, 'ac'); // Error! First parser succeeded on 'a' then failed on 'c'.
// Returned error is the error from the first parser

parse.attempt(p)

Parser that attempts to parse p and can backtrack if needed.

Useful with either parsers

// Modified example from 'parse.either'
var p = parse.either(
    parse.attempt(parse.next(
        parse_text.character('a'),
        parse_text.character('b'))),
    parse.next(
        parse_text.character('a'),
        parse_text.character('c')));

parse.run(p, 'ab'); // b
parse.run(p, 'ac'); // c
parse.run(p, 'z'); // Error! MultipleError 

parse.look(p)

Parse p, but don't consume any input if it succeeds. Returns results from p. This is the same as Parsec's lookahead

var p = parse.sequence(
    parse_text.character('a'),
    parse.look(parse_text.character('b')));

parse.run(p, 'ab'); 'b'
parse.run(p, 'ax'); Error
parse.run(p, 'a'); Error

// When look succeeds, it consumes no input
parse.run(parse.next(p, parse.anyToken), 'ab'); 'b'

parse.lookahead(p)

Like parse.look, but preserves user state after parsing p. position and input are still reverted.

parse.choice(...choices)

Attempts a variable number of parsers in order until one succeeds, or all fail. Returns result of first to succeed.

When all fails, returns a MultipleError with all errors from choices.

var p = parse.choice(
    parse_text.character('a'),
    parse_text.character('b'),
    parse_text.character('c')); 
parse.run(p, 'a'); // a
parse.run(p, 'b'); // b
parse.run(p, 'c'); // c
parse.run(p, 'z'); // Error! MultipleError

parse.choicea(choices)

Same as choice, but gets choices from array:

var p = parse.choicea(
    ['a', 'b', 'c']
        .map(parse_text.character));

parse.run(p, 'a'); // a
parse.run(p, 'b'); // b
parse.run(p, 'c'); // c
parse.run(p, 'z'); // Error! MultipleError

parse.choices(choices)

Same as parse.choice but get parsers as a [Nu][nu] stream. This allows parsing infinite lazy sequences of parsers.

parse.optional(default, p)

Parser p once, or return default if p fails.

var p = parse.optional('def', parse_text.character('b'));
parse.run(p, 'b'); // b
parse.run(p, ''); // 'def'
parse.run(p, 'z'); // 'def'

Enumeration

parse.many(p)

Consumes p zero or more times and succeeds with a [Nu][nu] stream of results.

It is a ParserError to run many with a parser the succeeds but consumes no input.

var p = parse.many(parse_text.character('a'));
parse.run(p, ''); // empty stream 
parse.run(p, 'z'); // empty stream
parse.run(p, 'a'); // stream of ['a'] 
parse.run(p, 'aaa'); // stream of ['a', 'a', 'a'] 
parse.run(p, 'aabaa); // stream of ['a', 'a'] 

parse.many1(p)

Parser that consumes p one or more times and succeeds with a [Nu][nu] stream of results.

var p = parse.many1(parse_text.character('a'));
parse.run(p, ''); // Error!
parse.run(p, 'z'); // Error!
parse.run(p, 'a'); // stream of ['a'] 
parse.run(p, 'aaa'); // stream of ['a', 'a', 'a'] 
parse.run(p, 'aabaa); // stream of ['a', 'a'] 

parse.cons(p, q)

Parser that cons the result of p onto result of q. q must return a [Nu][nu] stream.

var p = parse.cons(
   parse_text.character('a'),
   parse.enumeration(
      parse_text.character('b'),
      parse_text.character('c'));

parse.run(p, ''); // Error!
parse.run(p, 'z'); // Error!
parse.run(p, 'ab'); // Error!
parse.run(p, 'abc'); // stream of ['a', 'b', 'c'] 
parse.run(p, 'abcxyz'); // stream of ['a', 'b', 'c'] 

parse.append(p, q)

Parser that joins the result of p onto result of q. Both p and q must return [Nu][nu] streams.

var p = parse.append(
   parse.enumeration(
      parse_text.character('a'),
      parse_text.character('b'),
   parse.enumeration(
      parse_text.character('c'),
      parse_text.character('d'));

parse.run(p, ''); // Error!
parse.run(p, 'z'); // Error!
parse.run(p, 'ab'); // Error!
parse.run(p, 'abc'); // Error!
parse.run(p, 'abcd'); // stream of ['a', 'b', 'c', 'd'] 
parse.run(p, 'abcdefg'); // stream of ['a', 'b', 'c', 'd'] 

parse.enumeration(...parsers)

Consume parsers in order, building a [Nu][nu] stream of results.

var p = parse.enumeration
    parse_text.character('a'),
    parse_text.character('b'));
        
parse.run(p, 'ab'); // stream of ['a', 'b'] 
parse.run(p, 'ax'); // Error, expected b found x. No backtracking

parse.enumerationa(parsers)

Same as parse.enumeration but takes parsers an array.

parse.enumerations(parsers)

Same as parse.enumeration but takes parsers as a potentially infinite [Nu][nu] stream.

parse.eager(p)

Flattens the results of p to a Javascript array.

var p = parse.eager(parse.many(parse_text.character('a')));
parse.run(p, ''); // []
parse.run(p, 'z'); // []
parse.run(p, 'a'); //  ['a'] 
parse.run(p, 'aaa'); //  ['a', 'a', 'a'] 
parse.run(p, 'aabaa); //  ['a', 'a'] 

parse.binds(p, f)

Same operation as bind, but p succeeds with a [Nu][nu] stream and f is called with stream values as arguments.

var seq = parse.enumeration(parse.always(1), parse.always(2));

var p = parse.binds(seq, \x y -> parse.always(x + y));
parse.run(p, ''); // 3

var err = parse.binds(parse.never(), \x, y -> parse.always(x + y));
parse.run(p, ''); // throws UnknownError, 'f' never called.

Tokens

parse.token(consume, [err])

Parser that consumes a single item from the head of input if 'consume' returns true for that item. Fails to consume input if consume is false or input is empty.

When consume succeeds, advanced the input

'err' is called to get the error object when 'consume' fails. Defaults to returning an UnexpectError

var p = parse.token(\x -> x === 'a');
parse.run(p, ''); // Error!
parse.run(p, 'b'); // Error!
parse.run(p, 'a'); // 'a'
parse.run(p, 'abc'); // 'a'

var p = parse.next(
    parse.token(\x -> x === 'a'),
    parse.token(\x -> x === 'b'));
parse.run(p, ''); // Error!
parse.run(p, 'b'); // Error!
parse.run(p, 'a'); // Error!
parse.run(p, 'ab'); // 'b'
parse.run(p, 'abc'); // 'b'

var p = parse.token(x -> x === 'a', (pos, found) -> new parse.ExpectError(pos, 'a', found));
parse.run(p, ''); // Error! ExpectError for expected 'a'
parse.run(p, 'b'); // Error!  ExpectError for expected 'a'

parse.anyToken

Parser that consumes any token.

var p = parse.anyToken;
parse.run(p, ''); // Error! Unexpected eof
parse.run(p, 'b'); // 'b'
parse.run(p, 'a'); // 'a'

State Interaction

parse.getParserState

Succeeds with the current parse state.

parse.run(
     parse.next(
          parse_text.character('a'),
          parse.getParserState),
     'abc'); // returns a ParserState(Position(1), input, ud)

parse.setParserState(s)

Sets the parser state to s. Succeeds with s.

parse.modifyParserState(f)

Modify the state using function f, succeeding the result and setting the state to be the result.

parse.run(
     parse.modifyParserState(\s ->
           s.setPosition(parse.Position.initial)),
     'abc'); // returns the parserState

parse.extract(f)

Parser that extracts a value from the parser state by calling function 'f' called with state.

parse.getState

Succeeds with user state.

parse.run( parse.getState, "abc", 'user state'); // returns 'user state'

parse.setState(s)

Sets the user state to s. Succeeds with s.

parse.run(
    parse.setState('new user state'),
    "abc",
    'user state'); // returns 'new user state'

parse.modifyState(f)

Modify the users state using function f, succeeding the result and setting the state to be the result.

parse.run(
    parse.sequence(
         parse.modifyState(\x -> x + 10),
         parse.modifyState(\x -> x / 2),
         parse.getUserState)
    "abc",
    0); // returns 5

parse.getPosition

Get the current position.

parse.run(
    parse.next(
         parse_text.character('a'),
         parse.getPosition)
    "abc"); // returns Position(1)

parse.setPosition(pos)

Set the current position.

parse.getInput

Get the current input. This returns the stream of remaining input.

parse.run(
    parse.next(
         parse_text.character('a'),
         parse.bind(parse.getInput, \inputStream ->
              always(stream.toArray(inputStream)))
    "abc"); // returns ['b', 'c']

parse.setInput(input)

Set the current input to the stream input. Parsing continues on new input.

parse.run(
    parse.sequence(
         parse_text.character('a'),
         parse.setInput(stream.NIL),
         parse_text.character('b'))
    "abc"); // throws error, found eof expected 'b'

Objects

parse.Position

Default object used to track the parser's position in the input stream. 'parse.Position' simply keeps track of the index in the stream.

parse.InputState

Default state object. Keeps track of the stream and position.

Parser Creation

parse.rec(def)

Creates a function using a factory function 'def' to allow self references.

'def' is a function that is passed a reference to the object to be created and returns the object to be created.

For example, using a traditional definition the self reference to 'b' evaluates to undefined:

var b = parse.either(parse_text.character('b'), b)

// Really this is equivalent to:
var b = parse.either(parse_text.character('b'), undefined)

Using rec, we fix this.

var b = rec(\self ->
    parse.either(parse_text.character('b'), self));

and now 'b' correctly references itself.

parse.label(name, impl)

Create a parser with display name name and implementation impl. Display names help with debugging.

Structures

parse.Parser(impl)

Base parser type. All Bennu parsers are instances of this object.

You will probably never need to use this object directly, but all base parsers are instances of this object and it implements the [Fantasy Land][fl] methods.

Error Objects

ParserError(message)

Error thrown when there is an error with the parser definition itself (for example, calling many on a parser that succeeds and consumes no input).

ParseError(position, message)

Base type of error thrown durring during parsing.

The position property gets the location where the error occurred.

The message property gets the complete error description (you can also use toString).

The errorMessage property gets the description of just the error, without the position information.

MultipleError(...errors)

Merges one or more ParserError into a single error.

The position is the position of the first error. The message is combined messages from errors.

UnknownError(pos)

Error whose exact cause is unknown.

UnexpectError(pos, unexpected)

ParseError when an unexpected token 'unexpected' is encountered at position 'pos'.

ExpectedError(pos, expected, [found])

ParseError when an unexpected token 'found' is encountered at position 'pos' when 'expected was expected'.

[fl] : https://github.com/fantasyland/fantasy-land [nu] : https://github.com/mattbierner/nu