[BUG] - Rasa Spelling PLAYGROUND

[rmiaouh]

Hi guys & @koaning. Your work is amazing.

I have an issue with the Rasa Spelling PLAYGROUND

My model trained with : Rasa V2.1.3
RasaLit : V0.1.2

File "/home/robin/askMona_Projets/1_Projet_NomDeLieux/ArDf/Divers_Tests/TEST_RASA/env/lib/python3.6/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)
File "/home/robin/askMona_Projets/1_Projet_NomDeLieux/ArDf/Divers_Tests/TEST_RASA/env/lib/python3.6/site-packages/rasalit/apps/spelling/app.py", line 50, in <module>
    preds = clf.predict_proba(augs)
File "/home/robin/askMona_Projets/1_Projet_NomDeLieux/ArDf/Divers_Tests/TEST_RASA/env/lib/python3.6/site-packages/rasalit/apps/spelling/classifier.py", line 93, in predict_proba
    result.append([ranking_dict[n] for n in self.class_names_])
File "/home/robin/askMona_Projets/1_Projet_NomDeLieux/ArDf/Divers_Tests/TEST_RASA/env/lib/python3.6/site-packages/rasalit/apps/spelling/classifier.py", line 93, in <listcomp>
    result.append([ranking_dict[n] for n in self.class_names_])

---

KeyError: 'smalltalk residence'
Looks like it doesn't like how I called my Intent ?

[koaning]

I have a feeling that this might be related to your config.yml setting. Did you have a max-ranking set in DIET by any chance?

[rmiaouh]

I will send you my Metadata.txt. I dont think i have a max-ranking ?

metadata.txt

[koaning]

Ah yeah, I think I've found the issue. Your DIET classifier doesn't predict all possible values because of this setting in DIET;

"ranking_length": 10,

This might be a default value though so it's something that I need to fix on my end. So it's added to the TODO pile for this week. Thanks for reporting!

Will ping here one there's a fix.

[rmiaouh]

Oh thank you for all. Now I understand all the logic behind this bug.
I train models with 90 intents. Can this parameter impact the output performance of the model? Or this parameter only impacts the number of intents it displays ?

[koaning]

I'm not 100% sure yet, but I'll try to investigate this.

[koaning]

I've just pushed a PR with a fix, once tests are green it will be merged. Could you let me know if it works?

Today I learned that Rasa pipelines don't expose all the intent confidence scores on prediction. Only if you configure DIET to do so. The fix turned out to be simple and we're now also robust against the changes made to the confidence change.

In case you haven't seen the announcement yet.

[rmiaouh]

I've just pushed a PR with a fix, once tests are green it will be merged. Could you let me know if it works?
@koaning It's perfect ! Ty for all

May I ask you a last question ?
I currently have a problem with the DIET entity detection.
For exemple : if I train the entity "Livre" (means "book" in french) and I call the word "Live", the Diet entity detector will think it's the world "Livre".

And I think it's because of the CountVectors in my config :

• name: CountVectorsFeaturizer
  analyzer: "char_wb"
  min_ngram: 1
  max_ngram: 4

But from what I know the CountVectors is essential for better performance.
Do you have a strategy to avoid this problem?

Ty for all, I will watch the video :).

[koaning]

What entities are you trying to detect? Maybe a RegexEntityExtractor works better here.