A simple brainf**k interpreter made in rust.
-
Updated
Mar 16, 2023 - Rust
A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.
A simple brainf**k interpreter made in rust.
simple generic lexer / tokeniser, for use in rust programs and beyond
Converts Lox source code into syntax tokens.
From text input with indentation format, produce a flattened output extending char values. Good for iterating and parsing
Morphologically biased byte-pair encoding
Playing with logos: https://github.com/maciejhirsz/logos
Pretok is a pre-tokenizer for C-like syntaxes.