tokenizer
A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.
Here are 50 public repositories matching this topic...
.NET 7 SDK for OpenAI with a Blazor Server playground
-
Updated
May 17, 2023 - C#
A C# Lua, GLua and Luau parser, code analysis, transformation and generation library.
-
Updated
Mar 18, 2024 - C#
A collection of tools for working with OpenAI
-
Updated
May 4, 2023 - C#
The Parser Construction Kit ("Puck"): A Parser Generator and Grammar Translator in C#
-
Updated
Nov 24, 2019 - C#
.NET text parsing helper class.
-
Updated
Mar 17, 2024 - C#
A Unity package to run pretrained transformer models with Unity Sentis
-
Updated
Apr 10, 2024 - C#
#️⃣ C# Expression parser and evaluator, inspired from jokenizer project.
-
Updated
Feb 10, 2020 - C#
Semester project, the lexical analyzer for a custom compiler. Deterministic Finite Automaton is Implemented in this project.
-
Updated
Apr 7, 2019 - C#
A toolkit that helps you to write your own parser.
-
Updated
Mar 17, 2024 - C#
This application analyzes sentences. Normalizes sentences and separates them into words. It divides the words into their attachments. ITU uses NLP API for this.
-
Updated
Dec 8, 2022 - C#
A C# math expression parser and evaluator
-
Updated
Jun 18, 2021 - C#
This repository provides a lexer and parser for the Pascal programming language. Both lexer and parser can generate errors indicating the location in the code.
-
Updated
Jan 29, 2022 - C#
- Followers
- 10.1k followers
- Wikipedia
- Wikipedia