Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add exclusive but non-overlapping mode to tokens_lookup #502

Closed
koheiw opened this issue Jan 23, 2017 · 0 comments
Closed

Add exclusive but non-overlapping mode to tokens_lookup #502

koheiw opened this issue Jan 23, 2017 · 0 comments
Assignees
@koheiw
Labels
enhancement

Comments

@koheiw
Copy link
Collaborator

koheiw commented Jan 23, 2017

When I simply want to count the occurrences of dictionary entities, matching overlapping ones is sometimes harmful. For example, tokens_lookup counts 'Czech Republic' in a text twice with this:

- CZ: Czech Republic, Czech*, Prague
@koheiw koheiw self-assigned this Jan 23, 2017
@kbenoit kbenoit modified the milestone: CRAN refresh Jan 24, 2017
kbenoit added a commit that referenced this issue Jan 27, 2017
@kbenoit
Add more tests for #502
cb0cb93
@kbenoit kbenoit closed this as completed Feb 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@koheiw koheiw

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kbenoit @koheiw