Parser

Parser

Who this is for: parser engineers. Consumers need only the entry points and ParseOptions (also in API Reference); the exact early-error guard tag-sets and event-emission cursor live in Internals and Contributing.

parser.zig defines the Parser struct and the statement grammar; the expression grammar is in expressions.zig, the TS grammar in typescript.zig, and JSX in jsx.zig — all as pub fns operating on Parser. It is a hand-written recursive-descent parser with a Pratt (precedence-climbing) core for expressions.

Entry points

pub fn parse(allocator, source, tokens: TokenList.Slice) !Ast
pub fn parseWithOptions(allocator, source, tokens, opts: ParseOptions) !Ast
pub fn parseWithLanguage(allocator, source, tokens, language, is_module_file) !Ast
pub fn parseWithLanguageOpts(allocator, source, tokens, language, is_module_file, annex_b) !Ast

parse is the convenience entry: it calls parseWithOptions(.., .{ .emit_events = true }), so the returned Ast always carries scope events and the fast-path analyzer is available without extra arguments.

pub const ParseOptions = struct {
    language: Language = .js,        // .js .jsx .ts .tsx .dts
    is_module: bool = false,         // import/export + strict-mode semantics
    global_return: bool = false,     // parserOptions.ecmaFeatures.globalReturn
    is_strict: ?bool = null,         // override the strict mode implied by is_module
    annex_b: bool = true,            // web-compat extensions
    experimental_decorators: bool = false,  // legacy TS decorator placement rules
    events_out: ?*ScopeEventStream = null,  // emit into a caller-owned stream
    emit_events: bool = false,       // emit into the returned Ast.scope_events
    streaming: ?StreamingHooks = null,  // lex/parse/sem pipeline
};

is_strict defaults to is_module when left null. The full StreamingHooks shape is documented under Performance and Concurrency.

The driver

parseInternal constructs the Parser, pre-sizes its buffers (below), then parseProgram runs the top-level statement loop. The parser keeps hot lexer state as cached raw column pointers — tags_ptr, newlines_ptr, has_escape_ptr, tok_starts_ptr, tok_lens_ptr — instead of going through MultiArrayList.items(...) on every peek(). (tags_ptr and tok_starts_ptr are mutable because TS angle-bracket splitting rewrites tokens in place; see TypeScript and JSX.) Node-column pointers (node_tags_ptr, node_data_ptr, node_main_token_ptr) are likewise cached and refreshed whenever nodes grows.

parseStatement dispatches the statement grammar: blocks, if/else, the three loop families (while, do/while, for with its in/of/await of variants), switch, return/throw/break/continue (with and without labels), labeled statements, try/catch/finally, with, debugger, variable declarations (var/let/const), functions (plain/async/generator/async-generator), classes, the module forms (import, export, export default, export … from), and the TS declarations (interface, type, enum, namespace/module, declare). Each statement parser builds its node, records its end token, and (when emission is on) emits the corresponding scope/declare/control-flow events inline.

The Pratt expression engine

The core is parseExpressionPrec(p, min_prec) in expressions.zig:

1. parse a prefix expression (parsePrefixExpression): unary/update operators, await/yield, new, or a primary;
2. loop: look up the next token's precedence in the comptime prec_table ([256]Precedence indexed by tag); if it is below min_prec, stop;
3. otherwise dispatch — call-level (. ( [ ?. template), postfix ++/--, conditional ?:, comma, assignment, or a binary operator — and for binary operators recurse on the RHS at prec.next().

parseExpression enters at .comma; parseAssignmentExpression at .assignment. The call-level case (precedence 18) is the most common infix token and is handled with an inline fast path before the general dispatch.

Precedence table

Precedence is an explicit enum(u8); higher binds tighter:

Value	Level	Operators
0	none	(sentinel / non-operator)
1	comma	, (sequence)
2	assignment	= and all compound assigns (+= … ??=) — right-associative
3	conditional	?:
4	nullish_coalesce	??
5	logical_or	||
6	logical_and	&&
7	bitwise_or	|
8	bitwise_xor	^
9	bitwise_and	&
10	equality	== != === !==
11	relational	< > <= >= instanceof in
12	shift	<< >> >>>
13	additive	+ -
14	multiplicative	* / %
15	exponentiation	** — right-associative
16	unary	! ~ + - typeof void delete await (prefix)
17	postfix	++ -- (postfix)
18	call	( . [ ?. and tagged templates
19	new_expr	new
20	primary	literals, identifiers, this, super, (…)

Associativity is encoded in Precedence.next(): left-associative levels recurse at value + 1 (forcing the RHS to bind tighter → left grouping), while the two right-associative levels (assignment, exponentiation) recurse at the same level (→ right grouping). Notable correctness points enforced around the table:

• ?? cannot be mixed with &&/|| without parentheses — a syntax error in JS mode, checked on both operands. The guard is gated on !is_ts, so TS mode emits no syntax error here.

• ** rejects an un-parenthesized unary base (-2 ** 2 is an error; (-2) ** 2 and ++x ** 2 are fine) — matching the spec's UpdateExpression ** ExponentiationExpression. Also JS-mode only.

  (Exact guard tag-sets and messages: Internals.)

• in is suppressed in no-in contexts (e.g. a for header's init) via an allow_in flag, not via the precedence table.

• #x in obj (the private-in-check) is only valid as the LHS of in at relational precedence; a one-shot private_in_lhs_allowed flag, set in parseExpressionPrec from min_prec <= relational, gates the # primary.

Members, calls, optional chains, new

Member access allows keywords and escaped keywords as property names, and handles private fields (obj.#x, requiring # and the name to be contiguous), with validation that #x references occur inside a class and resolve to a declared private name. Optional chaining produces three dedicated tags (optional_member_expr, optional_computed_member_expr, optional_call_expr, with ?.#x reusing optional_member_expr); an optional chain may not be a new target, a direct tagged-template tag, or an assignment target. new parses its callee's member chain (./[/template) before arguments, handles new.target, and rejects new import(...) and bare new super().

Arrow vs. parenthesized disambiguation

This is the parser's main ambiguity. (a, b) could be a parenthesized expression or an arrow parameter list, and the deciding => (or a TS return-type :) can be far to the right. The parser uses a cover grammar with backtracking:

• A (…) group is parsed as comma-separated assignment expressions/spreads into scratch. If a => follows (on the same line — ASI applies), the scratch elements are reinterpreted as binding patterns (reinterpretAsPattern + validatePattern); otherwise it is a grouping_expr/sequence_expr.
• TS adds a wrinkle: a : after the params could be a return-type annotation or the alternate of an enclosing ternary. In a conditional context the parser speculatively parses the type annotation, checks for =>, and backtracks the annotation only if no => follows — snapshotting and restoring scratch, tokens, nodes, and extra_data.
• Single-identifier arrows (x => …) and generic/async generic arrows (<T>(…) =>, async <T>(…) =>) have their own speculative paths; the generic forms try to parse <…> as type parameters and restore parser state on failure.

While speculatively parsing a maybe-arrow's parameter list, suppress_param_declares prevents premature declare events; once the arrow is confirmed, the params' SubRange is walked and declares are emitted into the fresh arrow scope. Orphan reference events emitted speculatively are neutralized with nop events rather than removed.

Assignment-target and pattern validation

parseAssignment validates that the LHS is a valid assignment target: identifiers, member/computed-member expressions, and array/object destructuring patterns are valid; literals, this, super, calls (except the Annex B sloppy exception), optional chains, and most operator expressions are not. Strict mode additionally forbids assigning to eval/arguments. Destructuring targets are reinterpreted from the cover grammar and validated (no literals, no compound assignment inside patterns, well-formed rest elements).

ASI and context flags

Automatic semicolon insertion is handled at the points the spec requires it: postfix ++/-- must not be preceded by a newline (has_newline_before); return/throw/break/continue/yield end before a newline; arrow => must be on the same line as its parameter list. yield/await parse as operators only in generator/async contexts (and module top-level for await), and as identifiers otherwise, with the spec's early errors for their appearance in parameter defaults.

Error recovery

The parser is a panic-mode recoverer that always returns a usable Ast:

• expect(tag) consumes the token or, on a miss, emits a diagnostic naming the expected lexeme (cold path expectFail) and returns error.ParseError. eat(tag) is the non-erroring conditional consume.
• The top-level loop in parseProgram catches ParseError, increments a consecutive_errors counter (bailing once it exceeds 100, to bound runaway memory), calls synchronize(), guarantees forward progress by skipping one token if synchronize did not advance, and inserts an error_node placeholder into the statement list.
• synchronize() resynchronizes to the next statement boundary. It bulk-advances 16 token tags at a time with a SIMD comparison, stopping at ;, }, eof, or any statement-starting keyword, then finishes scalar. Some keyword stops are conservative (e.g. else) but only cause an early, harmless stop.
• error_node (main_token = error position) is the placeholder tag; the ESTree mapping calls it ErrorNode. Recovery is also used surgically — e.g. an invalid regex body emits a diagnostic but keeps the regex_literal node rather than collapsing to error_node (changelog 0.2.12), matching tsc.

Recursion and sizing limits

• Recursion depth. Expression/type/JSX nesting funnels through enterRecursion, which caps at max_recursion_depth = 400 (a pub const in parser.zig) and emits a "maximum nesting depth exceeded" diagnostic at the limit. This converts pathological deeply-nested input into a diagnostic instead of a stack overflow.
• Buffer pre-sizing. nodes and extra_data are pre-sized to @max(sizing_count * 3 / 4, 1) where sizing_count is tokens.len (or, in streaming mode, the caller's capacity_hint) — i.e. ~0.75 entries per token. The ref_event_idx and node_end_toks side arrays are sized to the resulting nodes.capacity (the rounded-up allocation, ≥ the estimate). max_nodes is a hard ceiling of sizing_count * 16 checked in addNode.

Inline event emission

When emit_scope_events is on, the statement and expression parsers call helper methods (emitScopeOpen/emitScopeClose, emitDeclare, emitReference, emitTerminator, and the branch/loop/try/switch/logical/cond/label/if helpers) that append to the EventStream as the tree is built. Reference-kind upgrades (e.g. read → read_write for x++) and cancellations are kept O(1) via a node-indexed cache. When emission is off, all these helpers compile to dead code. The emitted stream is what Semantic Analysis consumes; the write-cursor and cache mechanics are in Internals.

---

Next: TypeScript and JSX · Semantic Analysis