Skip to content

Commit

Permalink
version 2.0.1
Browse files Browse the repository at this point in the history
  • Loading branch information
stagas committed Jan 22, 2022
1 parent d7ac15b commit e648ad2
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 12 deletions.
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ iterator based tokenizer for writing parsers
· <a href="https://github.com/stagas/tokenizer-next/issues"> 🖐️ <strong>Help</strong></a>
</p>

---
***

## Install

Expand All @@ -37,15 +37,15 @@ See [TokenizerFactory](#tokenizerfactory).

#### Table of Contents

- [createTokenizer](#createtokenizer)
- [Parameters](#parameters)
- [TokenizerFactory](#tokenizerfactory)
- [Parameters](#parameters-1)
- [TokenizerCallableIterable](#tokenizercallableiterable)
* [createTokenizer](#createtokenizer)
* [Parameters](#parameters)
* [TokenizerFactory](#tokenizerfactory)
* [Parameters](#parameters-1)
* [TokenizerCallableIterable](#tokenizercallableiterable)

### createTokenizer

[src/index.ts:19-37](https://github.com/stagas/tokenizer-next/blob/c23a53a4be8e46b64d92477d01dea8f86d9c4eea/src/index.ts#L19-L37 'Source code on GitHub')
[src/index.ts:19-37](https://github.com/stagas/tokenizer-next/blob/d7ac15bc32521300c55c80a1b8f3952f47e81a67/src/index.ts#L19-L37 "Source code on GitHub")

Create a [TokenizerFactory](#tokenizerfactory) for the given RegExps.

Expand All @@ -60,11 +60,11 @@ const tokenize = createTokenizer(

#### Parameters

- `regexps` **...[Array](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array)<[RegExp](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/RegExp)>** RegExps to match.
* `regexps` **...[Array](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array)<[RegExp](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/RegExp)>** RegExps to match.

### TokenizerFactory

[src/index.ts:67-67](https://github.com/stagas/tokenizer-next/blob/c23a53a4be8e46b64d92477d01dea8f86d9c4eea/src/index.ts#L39-L66 'Source code on GitHub')
[src/index.ts:67-67](https://github.com/stagas/tokenizer-next/blob/d7ac15bc32521300c55c80a1b8f3952f47e81a67/src/index.ts#L39-L66 "Source code on GitHub")

Create a [TokenizerCallableIterable](#tokenizercallableiterable) for given input string.

Expand Down Expand Up @@ -95,11 +95,11 @@ Type: function (input: [string](https://developer.mozilla.org/docs/Web/JavaScrip

#### Parameters

- `input` The string to tokenize.
* `input` The string to tokenize.

### TokenizerCallableIterable

[src/index.ts:74-75](https://github.com/stagas/tokenizer-next/blob/c23a53a4be8e46b64d92477d01dea8f86d9c4eea/src/index.ts#L69-L73 'Source code on GitHub')
[src/index.ts:74-74](https://github.com/stagas/tokenizer-next/blob/d7ac15bc32521300c55c80a1b8f3952f47e81a67/src/index.ts#L69-L73 "Source code on GitHub")

Can be called to return next <a href="https://github.com/stagas/match-to-token#token">Token</a> or can be used as an
[Iterable](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterable_protocol)
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"author": "stagas",
"short": "stagas/tokenizer-next",
"description": "iterator based tokenizer for writing parsers",
"version": "2.0.0",
"version": "2.0.1",
"license": "MIT",
"repository": {
"type": "git",
Expand Down

0 comments on commit e648ad2

Please sign in to comment.