Use response selector keys for confusion matrix labelling

[hotzenklotz]

Proposed changes:

• At the moment, when running an NLU evaluation for response selectors the plotted confusion matrix with use the training data utterances as plot labels. This small PR uses the response selector keys and predicted (sub)-intents as labels instead. This make the confusion matrix a lot more readable and useful.
• Similarly, the response_selection_report.json uses the same labels leading to better comprehension.

Examples
Before

After

Status (please check what you already did):

• added some tests for the functionality
• updated the documentation
• updated the changelog (please check changelog for instructions)
• reformat files using black (please check Readme for instructions)

[sara-tagger]

Thanks for submitting a pull request 🚀 @tttthomasssss will take a look at it as soon as possible ✨

[dakshvar22]

Thanks for fixing this on 1.10.x branch 👍 I suggested a few changes and based on them there are a few more places where you will have to make changes. For example, line 412-414(return actual and predicted labels instead of actual and predicted texts), line 1488(use actual and predicted labels instead of actual and predicted texts to compute metrics), line 241(not filter response examples based on empty texts).
All of these changes are actually implemented on master already if you would like to view them for reference.

[dakshvar22]

I would suggest not splitting the predicted retrieval_intent/sub_intent because you could have same sub_intent under multiple retrieval intents. So, let's compare for e.g. faq/ask_name to faq/ask_weather and not ask_name to ask_weather.

[dakshvar22]

Following from the above comment, this would then change to example.get_combined_intent_response_key()

[dakshvar22]

It looks like with this change we can also get rid of intent_target, response_target and response_prediction. Can you scrub those off as well? There will be some more places where you will have to make changes.

[hotzenklotz]

@dakshvar22 Thanks for the feedback. I applied your suggestions. Please see commit 021bcb1

[hotzenklotz]

@dakshvar22 Is anything else required for this PR?

[dakshvar22]

Thanks for fixing this and addressing all my comments. ✨

[hotzenklotz]

@dakshvar22 To my dismay I discovered that my refactoring reintroduced a bug in my original PR. Unfortunately, it was captured by any of the unit the tests. I came across it when running a new response selector evaluation today. Please see this commit for a fix: hotzenklotz@fd037be

In short, the default value for a response selector key was wrong. It needs to default to None for all "regular" intent examples or otherwise those won't be filtered out further down the pipeline and crash the sklearn reports.

cc @tmbo