Handle custom fields from an NLU Message

[akelad]

A forum user (post) brought up a good point that at the moment if you build a custom NLU component, you have to put any additional information (like sentiment) into the entities key of the Message for the formation to be passed onto the dialogue manager.

A sentiment component right now should return output like this:

{
'intent':

{ 'name':'greet', 'confidence':0.44503513568867775 }

,
'entities':[

{ 'value':'neg', 'confidence':0.9933702940854111, 'entity':'sentiment', 'extractor':'sentiment_extractor' }

}

It would be more intuitive to have this information in a separate field since sentiment isn't really an entity:

{
"intent":

{ "name": "foo", "confidence": 0.1234567890 }

,
"entities": [],
"sentiment": [

{ "value": "NEUTRAL", "score": 0.123, "confidence": 0.897 }

],
}

On the NLU side the Message class would already allow this extra field (only on the master branch of rasa though) - see here. But I'm not sure whether the dialogue manager will handle that properly.

@degiz said he would look into this when he's back from vacation

[degiz]

Exalate commented:

degiz commented:

I think Initially I underestimated the situation here

😄

SentimentAnalyzer is a great example to think about!

1. Now, when we have YAML format with the metadata key, there's no need in having an additional labels.txt. The labels positive / negative / neutral can be stored just next to the training examples themselves. What's missing here is a "reader" methods for this metadata. I think we can easily implement that.

2. About the entities VS custom fields. Once the SentimentAnalyzer trained the model, and any incoming message from the user can be classified to have a certain sentiment, we want to write stories/rules, that will take this sentiment into the account. That's a classical categorical slot situation in my opinion. And we probably want the slot to be auto filled, once the sentiment was extracted from the user's utterance. So functionally the sentiment will be processed the same way as an entity.

In general I agree, that the sentiment is not an entity. But processing-wise, it's the same. So even if we store the sentiment as a separate field in a Message, we still will convert it to an entity (or reuse the same code as for entities). But that would also require some new configuration, because we'll have to let Core know that a certain field in a Message class is essentially an entity, and has to be treated that way.

I think that it makes a lot of sense to implement point 1 from above, and point 2 is OK as it is, unless there are other concerns.

@akelad wdyt?

[akelad]

Exalate commented:

akelad commented:

So point 1 I agree, that's fine. About point 2, hm but would the parsed message still come through as something like this?

{
"intent":

{ "name": "foo", "confidence": 0.1234567890 }

,
"entities": [],
"sentiment": [

{ "value": "NEUTRAL", "score": 0.123, "confidence": 0.897 }

],
}

I'm not sure we should process that the same way as an entity in terms of featurization. If people would want to store this info in a slot they can do that from a custom action no? If they want to do anything fancier then that, could they not just write a custom policy or something?

[degiz]

Exalate commented:

degiz commented:

About point 2, hm but would the parsed message still come through as something like this?

It will not be saved to the tracker atm, the code is here

If we want to save something extra to the tracker, aside from intent and entities, we have to know the name of this "custom" field, or ask everyone to put their data under pre-defined cusom key, so that it can be accessed from the custom action.

If they want to do anything fancier then that, could they not just write a custom policy or something?

Sure! It was us who showed an example with putting the sentiment to the entities here, and I still think it's the fastest way to have it working. Custom policies are more difficult imo.

[akelad]

Exalate commented:

akelad commented:

we have to know the name of this "custom" field, or ask everyone to put their data under pre-defined cusom key, so that it can be accessed from the custom action.

Could you not just use the metadata field?

Sure! It was us who showed an example with putting the sentiment to the entities here, and I still think it's the fastest way to have it working. Custom policies are more difficult imo.

I mean the reason we did that is only because there's no other way at the moment

Do we have any other use cases apart from the sentiment one btw? Feel like we're getting too deep into the details of this one

[degiz]

Exalate commented:

degiz commented:

Ok let's leave Sentiments out

😃

Could you not just use the metadata field?

metadata is a property of a training example in the YAML format. Do you mean to pull it all the way to the custom actions? Or just use the name metadata as a key that users should use when providing some extra attributes from the NLU component?

[akelad]

Exalate commented:

akelad commented:

So what I meant in this situation is that UserMessage (from the channel definition, which is the one that gets put in the tracker) actually has a metadata field at the moment. But the tracker.latest_message method doesn't even let you access that right now, only the parse data. That's actually a separate issue as well, that to access any metadata you assign there you have to iterate through the tracker events.

But yeah so what I'm saying is that we use a key like metadata, where users could provide extra NLU attributes. And I guess that would then be accessible from tracker.latest_message.get("metadata"). Or we actually fill the UserMessage metadata field with that as well, kind of depends.

[akelad]

Exalate commented:

akelad commented:

i'm trying to think of other use cases for this apart from the SentimentAnalyzer though

🤔

[mjaycub]

Exalate commented:

mjaycub commented:

@degiz what priority should we assign to this issue?

[degiz]

Exalate commented:

degiz commented:

@mjaycub added the low one for now, doesn't seem to be burning.

[sync-by-unito]

➤ Maxime Verger commented:

💡 Heads up! We're moving issues to Jira: https://rasa-open-source.atlassian.net/browse/OSS.

From now on, this Jira board is the place where you can browse (without an account) and create issues (you'll need a free Jira account for that). This GitHub issue has already been migrated to Jira and will be closed on January 9th, 2023. Do not forget to subscribe to the corresponding Jira issue!

➡️ More information in the forum: https://forum.rasa.com/t/migration-of-rasa-oss-issues-to-jira/56569.