A concrete implementation of tokens for Chearmyp language.
Add it to the dependencies:
[dependencies.chearmyp_token]
git = "https://github.com/KennethTrecy/chearmyp_token"
tag = "v1.0.0"
You may also activate all the features:
[dependencies.chearmyp_token]
git = "https://github.com/KennethTrecy/chearmyp_token"
tag = "v1.0.0"
features = ["no_std", "assertable_token"]
It was in a repository with the lexer library of Chearmyp. Yet, it has been forked for better development of future libraries.
Some parts of the repository were based from filled_bare_metal branch of Feo Template.
Run the following on the console:
cargo doc --all-features --open
If you want to contribute, this repository should be initialized to adhere in Conventional Commits specification for organize commits and automated generation of change log.
- Node.js and NPM
- pnpm (optional)
By running the command below, all your commits will be linted to follow the Conventional Commits specification.
$ npm install
Or if you have installed pnpm, run the following command:
$ pnpm install
To generate the change log automatically, run the command below:
$ npx changelogen --from=[tag name or branch name or commit itself] --to=master
The tokens only contain the most important data of a Chearmyp node. For example, a simplex node can have no attachers but it cannot exist without the concept name. The list below gives specific details about each tokens.
- Scope level. This token represents the indention of the subsequent token(s).
- Simplex. As stated above, a simplex token must contain the concept name only because it is the most important data.
- Complex. Same with the simplex, the concept name of a complex is the most important because a complex node can have no attachers or contents.
- Attacher. The label and content of an attacher are both important to add information to the simplex or complex they are attaching to.
- Comment. Since there are two types of comment nodes, they also have distinction as tokens.
- Line comment. All of its content are important.
- Block comment. The block that the block comment node contains is important.
- Othertongue. Othertongue tokens are almost the same as the comment tokens. Their only difference
is that they are intended to represent a foreign concept.
- Line othertongue
- Block othertongue
Consider the following Chearmyp text:
hello
name: ABC
# DEF
The token queue will represent the text as series of tokens. It will have a complex token (hello),
scope level token (with a value of 1), an attacher token (with name label and ABC content), and
a line comment token (with a content of DEF) in that order.
The repository is licensed under MIT.
Read the contributing guide for different ways to contribute in the project.
Chearmyp Token was created by Kenneth Trecy Tobias.