integrate chatterbot with spacy instead of nltk?

[brightening-eyes]

hi,
spacy is a very fast library for nlp which has all the features of nltk.
it is written in cython which make it so much faster.
it also has support for 49 languages, and more can be implemented.
why not use it instead of nltk?

[Orfeous]

Hello,

It's in progress and some parts of the core already using spacy see more in #1684

Also releases/1.1.0a1

[brightening-eyes]

what about stop words etc?
because spacy has these

[Orfeous]

it's not my call, I'm just a user :)

@gunthercox is the author, he could answer your question.

[gunthercox]

I agree, Spacy has a number of advantages. I'm aiming to leverage it's capabilities in the 1.1.x release series of ChatterBot.

[brightening-eyes]

maybe for BestMatch, we can use word vectors to find the match.
it is more reliable.

[brightening-eyes]

also for loading models, It should be possible to load the models explicitly. not just with a language model. for example, we can use "language_model" in the **kwargs argument to load the language model. because I'm using PyInstaller to make an executable, and It can't use the shortcut links, instead it can use packages without any problem. for example, "en_core_web_lg" instead of only "en"

[MohamedA95]

I agree, Spacy has a number of advantages. I'm aiming to leverage it's capabilities in the 1.1.x release series of ChatterBot.

How can we help ?

[gunthercox]

@habi3000 The changes are mostly finished. At this point I'm just waiting for various bugs related to the changes to be ironed out, such as #1736.

[brightening-eyes]

@gunthercox instead of loading the spacy models with spacy.load, i recommend to load them with imports to make them available to PyInstaller and such applications for making executables. spacy.load('en') loads the module with a shortcut link which is not support on PyInstaller and it is easy to change them in chatterbot.