Skip to content

Commit

Permalink
docs(*): update nlp utils description
Browse files Browse the repository at this point in the history
  • Loading branch information
sanjayaksaxena committed Oct 14, 2017
1 parent 7c83653 commit ffa7d18
Show file tree
Hide file tree
Showing 45 changed files with 89 additions and 89 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@

# wink-nlp-utils

> Easily tokenize, stem, phonetize, remove stop words, manage elisions, create ngrams, bag of words and more
> NLP Functions for amplifying negations, managing elisions, creating ngrams, stems, phonetic codes to tokens and more.
### [![Build Status](https://api.travis-ci.org/winkjs/wink-nlp-utils.svg?branch=master)](https://travis-ci.org/winkjs/wink-nlp-utils) [![Coverage Status](https://coveralls.io/repos/github/winkjs/wink-nlp-utils/badge.svg?branch=master)](https://coveralls.io/github/winkjs/wink-nlp-utils?branch=master) [![Inline docs](http://inch-ci.org/github/winkjs/wink-nlp-utils.svg?branch=master)](http://inch-ci.org/github/winkjs/wink-nlp-utils) [![dependencies Status](https://david-dm.org/winkjs/wink-nlp-utils/status.svg)](https://david-dm.org/winkjs/wink-nlp-utils) [![devDependencies Status](https://david-dm.org/winkjs/wink-nlp-utils/dev-status.svg)](https://david-dm.org/winkjs/wink-nlp-utils?type=dev)

[<img align="right" src="https://decisively.github.io/wink-logos/logo-title.png" width="100px" >](http://wink.org.in/)

Prepare raw text for Natural Language Processing (NLP) using **`wink-nlp-utils`**.It is a part of _[wink](http://wink.org.in/)_ — a growing family of high quality packages for Statistical Analysis, Natural Language Processing and Machine Learning in NodeJS.

It offers a set of **[APIs](#apis)** to work on **[strings](#string)** such as names, sentences, paragraphs and **[tokens](#tokens)** represented as an array of strings/words. They perform the required pre-processing for many simple ML tasks such as [semantic search](https://www.npmjs.com/package/wink-bm25-text-search), and [classification](https://www.npmjs.com/package/wink-naive-bayes-text-classifier).
It offers a set of **[APIs](wink.org.in/wink-nlp-utils/)** to work on **[strings](http://wink.org.in/wink-nlp-utils/#string)** such as names, sentences, paragraphs and **[tokens](http://wink.org.in/wink-nlp-utils/#tokens)** represented as an array of strings/words. They perform the required pre-processing for many simple ML tasks such as [semantic search](https://www.npmjs.com/package/wink-bm25-text-search), and [classification](https://www.npmjs.com/package/wink-naive-bayes-text-classifier).


## Installation
Expand Down
2 changes: 1 addition & 1 deletion docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -2333,7 +2333,7 @@ <h3 class='fl m0' id='stringtokenize'>
</div>


<p>The function uses the following set of rules to tokenize: </p>
<p>The function uses the following set of rules to tokenize:</p>
<ol>
<li>Single quotes are processed first as they may be part of elisions; and
<code>...</code> are converted to ellipses.</li>
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "wink-nlp-utils",
"version": "1.6.0",
"description": "Natural Language Processing Utilities that let you tokenize, stem, phonetize, create ngrams, bag of words and more.",
"description": "NLP Functions for amplifying negations, managing elisions, creating ngrams, stems, phonetic codes to tokens and more.",
"keywords": [
"Tokenize",
"Stem",
Expand Down
4 changes: 2 additions & 2 deletions src/helper-return-indexer.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/helper-return-quoted-text-extractor.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/helper-return-words-filter.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/name_cleaner_regexes.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// Easily tokenize, stem, phonetize, remove stop words,
// manage elisions, create ngrams, bag of words and more
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/phonetize_regexes.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// Easily tokenize, stem, phonetize, remove stop words,
// manage elisions, create ngrams, bag of words and more
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-amplify-not-elision.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-bong.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// bongs, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-compose-corpus.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-extract-persons-name.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-extract-run-of-capital-words.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-lower-case.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-marker.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-ngram.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-phonetize.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-remove-elisions.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-remove-extra-spaces.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-remove-html-tags.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-remove-punctuations.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-remove-spl-chars.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-retain-alpha-nums.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-sentences.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-soc.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-song.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// songs, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-soundex.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-split-elisions.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-stem.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
6 changes: 3 additions & 3 deletions src/string-tokenize.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down Expand Up @@ -31,7 +31,7 @@ var rgx = require( './util_regexes.js' );
// ### tokenize
/**
*
* The function uses the following set of rules to tokenize:
* The function uses the following set of rules to tokenize:
*
* 1. Single quotes are processed first as they may be part of elisions; and
* `...` are converted to ellipses.
Expand Down
4 changes: 2 additions & 2 deletions src/string-tokenize0.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-trim.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/string-upper-case.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-append-bigrams.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, appendBigramsming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE SyappendBigramss Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-bigrams.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, bigramsming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Sybigramss Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-bow.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// bows, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-phonetize.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, phonetizeming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Syphonetizes Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-propagate-negations.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// propagateNegationss, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-remove-words.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, phonetizeming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Syphonetizes Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-soundex.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, phonetizeming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Syphonetizes Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-sow.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// bows, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
4 changes: 2 additions & 2 deletions src/tokens-stem.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// wink-nlp-utils
// NLP Functions for removing HTML Tags, Managing Elisions,
// NGrams, Stemming, Phoneticising to Tokenizating and more.
// NLP Functions for amplifying negations, managing elisions,
// creating ngrams, stems, phonetic codes to tokens and more.
//
// Copyright (C) 2017 GRAYPE Systems Private Limited
//
Expand Down
Loading

0 comments on commit ffa7d18

Please sign in to comment.