Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Add word boundary check to period tokenizer regex. #2

Merged
merged 1 commit into from Sep 28, 2012
Commits on Sep 28, 2012
  1. Add word boundary check to period tokenizer regex.

    fschaefer committed Sep 28, 2012
    When only a millisecond based period is provided (e.g. "45 milliseconds"), the period is recognized as minutes ("45m").
    This commit adds a word boundary check to the tokenizer regex to distinguish between minutes and milliseconds.