How to use LUIS docker containers with Speech SDK

[rayyan808]

Hello,

I have setup 3 Docker containers on my Azure VM. A Speech to Text, Text to Speech and LUIS container. They are all working fine, I am able to access them from their routed ports through my domain, I am able to open the swagger documentation aswell.

On my Mobile Application (iOS and Android), I have setup an SPXSpeechRecognizer and SPXSpeechSynthesizer, assigned them to these custom endpoints and have gotten them running successfully. However, I am having issues with the LUIS container. What I am doing:

speechRecognitionConfig = try SPXSpeechConfiguration.init(host: "ws://onpremdev.westeurope.cloudapp.azure.com:8000");
speechSynthesisConfig = try SPXSpeechConfiguration.init(host: "http://onpremdev.westeurope.cloudapp.azure.com:9000");
speechConfigNLU = try SPXSpeechConfiguration.init(host: "ws://onpremdev.westeurope.cloudapp.azure.com:8000");

I then create an SPXIntentRecognizer, and load in my LUIS setup (LUIS is on port 7000 as follows:

intentRecognizer = try SPXIntentRecognizer(speechConfiguration: speechConfigNLU!, audioConfiguration: audioConfiguration!);
let intentModel = SPXLanguageUnderstandingModel.init(endpoint: "http://onpremdev.westeurope.cloudapp.azure.com:7000")

This does not work, the intent is never recognized. I am able to use the swagger on the LUIS container to successfully get an intent recognition, so I know it is setup correct (Loaded the correct .gz file and mounted it to the docker image /input. What I have also tried is:

SPXLanguageUnderstandingModel.init(endpoint: "http://onpremdev.westeurope.cloudapp.azure.com:7000/luis/v2.0/apps/{appId}")

I figured maybe it is my setup, so I decided to directly input my LUIS portal application prediction endpoint which also does not work within the context of the SDK:

let intentModel = SPXLanguageUnderstandingModel.init(endpoint: "https://ascenscia-prediction-luis.cognitiveservices.azure.com/luis/prediction/v3.0/apps/{app_id}/slots/production/predict?verbose=true&show-all-intents=true&log=true&subscription-key={SUS_KEY_SECRET}")

and other variations:

let intentModel = SPXLanguageUnderstandingModel.init(endpoint: "https://ascenscia-prediction-luis.cognitiveservices.azure.com/luis/v2.0/apps/{appId}?")

let intentModel = SPXLanguageUnderstandingModel.init(endpoint: "https://ascenscia-prediction-luis.cognitiveservices.azure.com/")

What is the correct way to load an endpoint into an SPXLanguageUnderstandingModel to use with an IntentRecognizer within the iOS Speech Service SDK?

Tried on Version 1.25.0, then upgraded to latest. No changes unfortunately.
: Installing MicrosoftCognitiveServicesSpeech-iOS 1.35.0 (was 1.25.0)

[rayyan808]

I would also like to add that there are no errors thrown when loading the endpoint inside SPXLanguageUnderstandingModel, and no methods or variables exposed by this object for me to debug any further what the issue is. This would be a nice feature to have in the next release. There are also no logs on my LUIS container when the SDK runs, though I will get logs when I send API requests via curl or the containers swagger.

[glecaros]

Hi @rayyan808,

You can't use the SDK for SR+LUIS with containers. The service only does the orchestration when hitting the azure endpoints, so in this case you would need to use the SpeechSDK for the speech container and the LUIS SDK for intent recognition, but you would need to orchestrate yourself.

See #424 for more info.