Stanford Tokenizer works, filtered tokens < 3 chars #2

Merged
merged 1 commit into from Jan 30, 2013

Conversation

Projects
None yet
2 participants
@rjurney
Contributor

rjurney commented Jan 29, 2013

It works but was adding some shitty punctuation tokens.

thedatachef added a commit that referenced this pull request Jan 30, 2013

Merge pull request #2 from rjurney/master
Stanford Tokenizer works, filtered tokens < 3 chars

@thedatachef thedatachef merged commit ab83f22 into thedatachef:master Jan 30, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment