Getting Error where running the doc code

[aswin-giridhar]

[tsproisl]

Unfortunately I'm not able to reproduce that error:

>>> from somajo import Tokenizer, SentenceSplitter
>>> tokenizer = Tokenizer(language="en")
>>> tokenizer.tokenize("Here's a simple example!:-)")
['Here', "'s", 'a', 'simple', 'example', '!', ':-)']

Which versions of SoMaJo and of the regex module are you using? For me everything works fine using SoMaJo 1.9.0 and regex 2.5.29 on either Python 3.5.2 or Python 3.7.3.

[aswin-giridhar]

Hi,
The versions are as follows:
regex==2017.11.9
SoMaJo==1.9.0
Python 3.6.3
Could you tell me which regex package did you use?
Thanks for taking your time to look into this issue.

[tsproisl]

I used version 2019.4.14 of the regex module (which self-reports as 2.5.29):

>>> regex.__version__
'2.5.29'

But even with version 2017.11.9 (self-reported as 2.4.132) everything works as expected.

Does this minimal working example work for you? What does it output (including complete stack trace, if any)? For me it looks like this:

regex: 2.4.132
somajo: 1.9.0
python: 3.5.2 (default, Nov 12 2018, 13:43:14) 
[GCC 5.4.0 20160609]
default encoding: utf-8
['Here', "'s", 'a', 'simple', 'example', '!', ':-)']