Yet another powerful tokenizer in js.
-
Updated
Jun 30, 2017 - JavaScript
Yet another powerful tokenizer in js.
Generate string from a token array by interpolating values.
Create a snapdragon token. Used by the snapdragon lexer, but can also be used by plugins.
Uses babel to extract JavaScript code comments from a string. Returns an array of comment objects, with line, column, index, comment type and comment string.
Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.
Extract JavaScript code comments from a string or glob of files.
Tokenize2 is a plugin which allows your users to select multiple items from a predefined list or ajax, using autocompletion as they type to find each item. You may have seen a similar type of text entry when filling in the recipients field sending messages on facebook or tags on tumblr.
NLP Functions for amplifying negations, managing elisions, creating ngrams, stems, phonetic codes to tokens and more.
Add a description, image, and links to the tokenize topic page so that developers can more easily learn about it.
To associate your repository with the tokenize topic, visit your repo's landing page and select "manage topics."