PR: integrate padatious NER and intent parser into Rasa framework

[repodiac]

Description of Problem:
The existing intent and/or NER parsers in Rasa work not so well with little data. Especially NER is difficult with the existing toolbox. For proof-of-concept or rapid prototyping as well as "small" development projects it would be great to have easy-to-use and more or less out-of-the-box solutions which can be adapted by the user/developer directly.

Overview of the Solution:
I would like to integrate a PR which makes use of padatious, an intent parser from the well-known project MyCroft. It uses a NN architecture for intent parsing which is already very acurate with a small number of intent examples and uses regular expression for NER - this makes it fast, easy to adapt and very useful for the task, in my opinion.

So the request is to integrate padatious as an additional combined NER and intent parser into the Rasa framework so that people can choose if they need these features.

Examples (if relevant):
References:
https://mycroft-ai.gitbook.io/docs/mycroft-technologies/padatious
https://github.com/MycroftAI/padatious

Blockers (if relevant):
none that I know of

Definition of Done:

• Code merged from fork
• Tests are added
• Feature described in the docs
• Feature mentioned in the changelog

[sara-tagger]

Thanks for submitting this feature request 🚀 @koaning will get back to you about it soon! ✨

[koaning]

Just from glancing at it is seems that padatious depends on FANN, a neural network library that did not receive an update for nearly two years. It might be interesting to see what this library can offer, but I'm a little uneasy about adding support for tools that seem inactive. Have you run any serious benchmarks using this tool?

That said, there's a new feature in Rasa 2.0 that also allows you to define regexes that detect named entities. Depending on the language that you're using you might also be able to use custom Matcher models in spaCY in Rasa.

[repodiac]

I understand. It does not seem to be the most fancy NN-Toolkit on the planet, but it works really well! What you mention as an alternative is not really a full substitute, you do neither have to come up with a regex yourself (done by padatious) nor to really train on lots of data -- as said, really a few examples suffice and results are really good compared to this.

I didn't benchmark yet, but used it in my own project where it worked just perfect for my understanding. The code (padatious) is a bit hard to read and it's not clear if the original developers are willing to change that... so reimplementing the FANN model in TF or similar may sound like a great idea, but probably takes a lot of efforts.

I can only recommend to try out for yourself - it really beats the existing Rasa tooling in my opinion for this matter (small amount of data, works out of the box, high accuracy).

[koaning]

I'm interested in exploring the tool, but it seems there's too many issues for it to support direct here. The padatious requirements.txt file suggests that it using FANN2 as opposed to FANN which is listed on the mycroft docs page. Inspecting the pypi repo of that project suggest only python 2.6-3.6 are supported while here at Rasa we also support 3.7, 3.8.

I've created a ticket over at the examples repo so that I have it in the back of my mind for experimentation. For now though, it seems like we cannot support the tool directly here.

[repodiac]

OK, I understand. Fair enough. If I have some news (maybe I once try to reimplement it in TF or similar) I will get back to you.