Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UTF-8 and Optionals #37

Closed
kirsle opened this issue Aug 4, 2016 · 2 comments
Closed

UTF-8 and Optionals #37

kirsle opened this issue Aug 4, 2016 · 2 comments

Comments

@kirsle
Copy link
Member

kirsle commented Aug 4, 2016

This might be related to rivescript-js#147. The following test case produces the problem:

+ *
- Häää?

+ [*] (überrasch mich|empfiehl mir was|empfehl mir was|was gibts neues für mich|empfehlung) [*]
- recommendation

The words with umlauts in them make the second trigger unmatchable.

In rivescript-js#147 I found Python wasn't affected by the Chinese symbols but may be having problems with these umlauts for some reason.

@kirsle kirsle added the bug label Aug 4, 2016
@Xiangshen-Meng
Copy link

Xiangshen-Meng commented Aug 30, 2016

Maybe the Chinese symbols problem is caused by space.
https://github.com/aichaos/rivescript-python/blob/master/rivescript/brain.py#L449
For language doesn't separate by space(Like Chinese), this line may cause problem.

p = r'(?:\\s|\\b)+{}(?:\\s|\\b)+'.format(p)

How about change it to this?

p = r'(?:\\s|\\b)?{}(?:\\s|\\b)?'.format(p)

But this may cause the English in word optionals.

Sorry I am not familiar with RiveScript and this is only my thoughts.

@kirsle kirsle added the unicode label Dec 16, 2016
htdinh pushed a commit to htdinh/rivescript-python that referenced this issue Sep 21, 2017
htdinh pushed a commit to htdinh/rivescript-python that referenced this issue Sep 21, 2017
kirsle added a commit that referenced this issue Sep 21, 2017
#37 Fix unicode handling with optionals
@kirsle
Copy link
Member Author

kirsle commented Sep 21, 2017

Fixed by PR #108

@kirsle kirsle closed this as completed Sep 21, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants