Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

add note about regexes

  • Loading branch information...
commit 12d34eb79f55ebbd7a7aa95fa5b1fe390866bfca 1 parent 02f8b19
Florent Jaby authored April 10, 2011

Showing 1 changed file with 4 additions and 0 deletions. Show diff stats Hide diff stats

  1. 4  README.md
4  README.md
Source Rendered
@@ -43,6 +43,10 @@ The tokenizer tries to find the longest string matching one or more rules.
43 43
 When several rules match the same string, priority is given to the rule
44 44
 which was added first. (this may change)
45 45
 
  46
+Please note that your regular expressions should use ^ and $ in order
  47
+to test the whole string. If these are not used, you rule will match _every_
  48
+string that contains what you specified, this could be the whole file!
  49
+
46 50
 ## To do
47 51
 * a lot of optimisation
48 52
 * being able to share rules across several tokenizer 

0 notes on commit 12d34eb

Please sign in to comment.
Something went wrong with that request. Please try again.