Dialogflow detects wrong intent with queries containing an apostrophe sent from Flutter app

[finnborchers]

Hello!

I coded a Flutter app which uses a Dialogflow API to send requests to my Dialogflow agent. Now I have the problem that queries sent from the Flutter app that have an apostrophe in it (e.g. I'm jogging) get a wrong intent match.
I already tried importing several Dialogflow APIs such as

https://pub.dev/packages/flutter_dialogflow_v2
https://pub.dev/packages/flutter_dialogflow

Both of them give the same result. So first of all, here is a screenshot of the intent I want my Flutter app to match when I type this: I'm jogging

On the right hand side you can see that matching this query works fine inside the Dialogflow Console.
And here is the raw api response when I click on diagnostic info below the "Try it now"-field inside the Dialogflow Console:

{ "responseId": "06e5b7c1-0b27-4f7d-bca9-23aa25575592-59c3eb0f", "queryResult": { "queryText": "I'm jogging", "parameters": { "pronounsIndefinite": [], "articles": [], "verbsgerund": [ "jogging" ], "nouns": [], "verbsPastParticiple": [], "verbshelppresent": [ "am" ], "adverbsTime": [], "pronounsdefinite": [ "I" ], "prepositions": [] }, "allRequiredParamsPresent": true, "fulfillmentText": "You are jogging, cool! ", "fulfillmentMessages": [ { "text": { "text": [ "You are jogging, cool! " ] } } ], "outputContexts": [ { "name": "projects/reddy-1/locations/global/agent/sessions/7e6ef20b-4de3-5758-ceca-a1996acf87f8/contexts/__system_counters__", "lifespanCount": 1, "parameters": { "prepositions": [], "prepositions.original": [], "no-match": 0, "verbsgerund": [ "jogging" ], "verbshelppresent": [ "am" ], "nouns": [], "verbshelppresent.original": [ "'m" ], "verbsPastParticiple.original": [], "pronounsIndefinite.original": [], "no-input": 0, "pronounsIndefinite": [], "nouns.original": [], "adverbsTime.original": [], "articles": [], "articles.original": [], "adverbsTime": [], "pronounsdefinite": [ "I" ], "verbsPastParticiple": [], "pronounsdefinite.original": [ "I" ], "verbsgerund.original": [ "jogging" ] } }, { "name": "projects/reddy-1/locations/global/agent/sessions/7e6ef20b-4de3-5758-ceca-a1996acf87f8/contexts/start", "lifespanCount": 9999, "parameters": { "articles": [], "sessionID_original": "3wd93kj4-2jeo-0d82-2hd8ej9f7jp1", "adverbsTime": [], "nouns": [], "articles.original": [], "verbshelppresent.original": [ "'m" ], "nouns.original": [], "verbsgerund": [ "jogging" ], "pronounsdefinite.original": [ "I" ], "prepositions.original": [], "pronounsIndefinite": [], "verbsgerund.original": [ "jogging" ], "verbsPastParticiple": [], "verbshelppresent": [ "am" ], "pronounsIndefinite.original": [], "verbsPastParticiple.original": [], "adverbsTime.original": [], "pronounsdefinite": [ "I" ], "prepositions": [] } } ], "intent": { "name": "projects/reddy-1/locations/global/agent/intents/176fc623-739e-4463-8df7-b970749630b2", "displayName": "gHealthFitnessAC" }, "intentDetectionConfidence": 1, "diagnosticInfo": { "webhook_latency_ms": 4153 }, "languageCode": "en" }, "webhookStatus": { "message": "Webhook execution successful" }, "agentId": "0bafa24f-02d0-46a3-ad37-b6011d756104" }

Now here is a screenshot of the intent that get's a wrong match when I send the query (I'm jogging) from the Flutter app that uses a Dialogflow API:

And here is the raw api response when I click on history and then the last conversation from the Flutter app with Dialogflow:

{ "id": "c0d19ce2-a638-4243-8612-56d00250bc13-59c3eb0f", "fulfillmentText": "gLeisureShoppingAC", "language_code": "en", "queryText": "I‘m jogging", "webhookPayload": {}, "intentDetectionConfidence": 1, "action": "", "webhookSource": "", "parameters": { "pronounsdefinite": [ "I" ], "articles": [], "prepositions": [], "verbsGerund": [ "jogging" ], "nouns": [], "verbsHelpPresent": [ "am" ], "adverbsTime": [] }, "fulfillmentMessages": [ { "text": { "text": [ "gLeisureShoppingAC" ] } } ], "webhookStatus": { "webhookUsed": false }, "outputContexts": [ { "lifespanCount": 1, "name": "__system_counters__", "parameters": { "adverbsTime.original": [], "no-input": 0, "pronounsdefinite.original": [ "I" ], "no-match": 0, "adverbsTime": [], "articles.original": [], "verbsGerund": [ "jogging" ], "prepositions.original": [], "pronounsdefinite": [ "I" ], "verbsGerund.original": [ "jogging" ], "verbsHelpPresent.original": [ "‘m" ], "nouns.original": [], "nouns": [], "verbsHelpPresent": [ "am" ], "articles": [], "prepositions": [] } }, { "lifespanCount": 9999, "name": "start", "parameters": { "nouns.original": [], "pronounsdefinite.original": [ "I" ], "sessionID_original": "3wd93kj4-2jeo-0d82-2hd8ej9f7jp1", "adverbsTime.original": [], "verbsGerund.original": [ "jogging" ], "articles": [], "verbsHelpPresent": [ "am" ], "prepositions": [], "adverbsTime": [], "pronounsdefinite": [ "I" ], "articles.original": [], "verbsHelpPresent.original": [ "‘m" ], "verbsGerund": [ "jogging" ], "prepositions.original": [], "nouns": [] } } ], "intent": { "isFallback": false, "displayName": "gLeisureShoppingAC", "id": "76efc477-006c-4ccd-a0c9-6e254ec72976" } }

All my intents have a start context. I am sure this does not influence the intent matching as all of my intents have the start context and I also removed all context and got the same result.

What I just noticed writing this Issue here:
When you look at both raw api responses, right at the beginning, there is the query text:

1. [...] "queryText": "I'm jogging"
2. [...] "queryText": "I‘m jogging",

I copied them directly and I noticed that the apostrophes look different (The second one is from the Flutter app and I typed in exactly the same query but there still is this difference). I don't know if this is a reason for the intent-mismatch.

Unfortunately I am not an expert, so I haven't tried making an http request through curl, Postman or Insomnia yet, as I am not sure how to do it. I solved all problems I had with my Flutter app and Dialogflow but this one got me stuck for a really long time now. I got it to this point where I know it has to do with the apostrophe in the query text that matches to the wrong intent.

Maybe it is because of the API I am using in my Flutter app that changes the apostrophe somehow.

I also found this related issue where other users seem to have a similar problem with apostrophes in their queries:
VictorRancesCode/flutter_dialogflow#21

Thank you for reading this issue and I hope we can find a solution together.

Greetings
Finn

[ElZombieIsra]

Hi!

For what you describe, I think the problem is within the json.encode function that we're using to send the info to the DialogFlow API.

I've found this answer related to an issue that I think would solve your problem.

Give me a couple of days to test this out and I'll let you know what I found.

[ElZombieIsra]

Try to use the solution proposed in the answer linked in the previous comment. I'll close this for now but we can re-open it if this is still an issue you'd like me to check into.