Permalink
Browse files

fix markdown

  • Loading branch information...
1 parent 1551ca5 commit 5787d8874ca0aa68576297b44e8defffa6ea3d9b @Floby committed Apr 14, 2011
Showing with 14 additions and 13 deletions.
  1. +14 −13 README.md
View
27 README.md
@@ -7,29 +7,30 @@ node-tokenizer is published on npm so you can install it with
## How to
* require the Tokenizer constructor
- var Tokenizer = require('tokenizer');
+ var Tokenizer = require('tokenizer');
* construct one (we'll see what the callback is used for)
- var t = new Tokenizer(mycallback);
+ var t = new Tokenizer(mycallback);
* add rules
- t.addRule(/^my regex$/, 'type');
+ t.addRule(/^my regex$/, 'type');
* write or pump to it
- t.write(data);
- // or
- stream.pipe(t);
+ t.write(data);
+ // or
+ stream.pipe(t);
* listen for new tokens
- t.on('token', function(token, type) {
- // do something useful
- // type is the type of the token (specified with addRule)
- // token is the actual matching string
- })
- // alternatively you can listen on the 'data' event
+ t.on('token', function(token, type) {
+ // do something useful
+ // type is the type of the token (specified with addRule)
+ // token is the actual matching string
+ })
+ // alternatively you can listen on the 'data' event
* look out for the end
- t.on('end', callback);
+ t.on('end', callback);
the optional callback argument for the constructor is a function that will
be called for each token in order to specify a different type by returning
a string. The parameters passed to the function are token(the token that we found)
and match, an object like this
+
{
regex: /whatever/ // the regex that matched the token
type: 'type' // the type of the token

0 comments on commit 5787d88

Please sign in to comment.