Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

词典不支持空格的分词 #13

Open
jianfengye opened this issue Jan 31, 2015 · 3 comments
Open

词典不支持空格的分词 #13

jianfengye opened this issue Jan 31, 2015 · 3 comments

Comments

@jianfengye
Copy link

词典载入LoadDictionary感觉有点问题:
1 支持没有词性标注(为什么需要支持没有词性标注,强制要求每个词语按照"分词文本 频率 词性" 的格式会省掉很多问题)
2 对于分词文本包含空格的情况没有支持
即,如果词典中为 my darling 10 n 就会解析错误
3 log.fatal 可能会导致某个应用程序跑到这里的时候直接断开了,作为第三方包最好还是不要使用fatal这样的函数,把错误返回给外层来处理会不会比较好~

@jianfengye
Copy link
Author

还有跳过的行没有log下来,这样即使错了某个行,分词不准找问题可能要找很久~

@suntong
Copy link

suntong commented Jul 4, 2016

👍

@PotiXIE
Copy link

PotiXIE commented Oct 13, 2016

image
修改这个地方,可以支持“分词文本包含空格的情况”。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants