Code example for the Fundamentals of Programming course at the University of Guilan, Fall 2016.
-
Updated
Jul 17, 2018 - C++
Code example for the Fundamentals of Programming course at the University of Guilan, Fall 2016.
A super string class for C++
Implementation of C++ lexical analyzer to demonstrate how it actually works as a part of the compiler.
A MIPS assembler that converts MIPS Assembly Language into MIPS Machine Language. Implemented DFA algorithms, Parsing, and Semantic Analysis.
Developed Console based API takes reviews from various mobile companies and compares the different aspects of the mobile phone using complex data structures concepts.
UAT es un tokenizer aritmético básico con funciones para determinar errores, separar tokens por tipos y preparar strings para conversiones a infix, etc.
🔧 Demonstration of using ANTLR4 (with runtime for C++) in projects for context-free grammar processing. The ANTLR4 (Java) package is included, and the project is configured to compile on Linux.
Smart Language Model
Originally a semester project for VT3574 Applied Software Design
This project is a fully functional compiler for the TINY programming language, which is a language that supports basic arithmetic, boolean, and control flow operations. The compiler can scan, parse, and run code written in the TINY language.
R package for Byte Pair Encoding based on YouTokenToMe
Fast and customizable text tokenization library with BPE and SentencePiece support
experimental programming language made with cpp
tf-idf (Term-Frequency Inverse-Document-Frequency) C++ Library
Unsupervised text tokenizer focused on computational efficiency
Lexical Analyzer for C made in C++
Includes coursework and lab materials for students enrolled in the Bachelor of Science in Computer Science degree at UBIT.
Tokenizers and Machine Learning Models for biological sequence data
The Libre Multilingual Analyzer, a Natural Language Processing (NLP) C++ toolkit.
Add a description, image, and links to the tokenization topic page so that developers can more easily learn about it.
To associate your repository with the tokenization topic, visit your repo's landing page and select "manage topics."