Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception when parsing ' ' with tokenize=False #134

Open
salty-horse opened this issue Nov 16, 2015 · 1 comment
Open

Exception when parsing ' ' with tokenize=False #134

salty-horse opened this issue Nov 16, 2015 · 1 comment

Comments

@salty-horse
Copy link

This throws an exception with pattern 2.6:

from pattern.en import parse
result = parse(' ', tokenize=False)

The last frame of the exception:

/pattern/text/__init__.pyc in _v(self, token, previous, next)
    310         v = {}
    311         f(v,  "b", "b")         # Bias.
--> 312         f(v,  "h", token[0])    # Capitalization.
    313         f(v,  "w", token[-6:] if token not in self.known or token in self.unknown else "")
    314         f(v,  "x", token[-3:])  # Word suffix.

IndexError: string index out of range
@salty-horse salty-horse changed the title Exception when parsing ' ' with tokenize=True Exception when parsing ' ' with tokenize=False Nov 16, 2015
@schwittlick
Copy link

Same here, how can this be approached?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants