Tokenizer
JavaScript
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
Makefile
Readme.md
component.json
index.js

Readme.md

Tokenizer

String tokenizer component

usage

var Tokenizer = require('tokenizer');

var tokenizer = new Tokenizer();

tokenizer.tokenize('viva la pepa    ');
  //['viva', 'la', 'pepa']

Methods

Tokenizer#trim

Tokenizer#tokenize