Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LUIS] Prompt response starts new dialog with LUIS recognizer. #2670

Closed
2 tasks done
dkushner opened this issue May 1, 2017 · 10 comments
Closed
2 tasks done

[LUIS] Prompt response starts new dialog with LUIS recognizer. #2670

dkushner opened this issue May 1, 2017 · 10 comments
Labels
customer-replied-to Indicates that the team has replied to the issue reported by the customer. Do not delete.

Comments

@dkushner
Copy link

dkushner commented May 1, 2017

Pre-flight checklist


System Information

  • Node SDK
  • v3.7.0
  • localhost
  • Node v7.6.0 and npm v4.2.0

Issue Description

Prompt in LUIS intent dialog attempts to start a new dialog rather than resume the suspended dialog that began the prompt.

http://stackoverflow.com/questions/43712861/how-can-i-properly-prompt-a-user-without-having-luis-attempt-to-recognize-the-re

Example Code

Bot Setup

const connector = new builder.ChatConnector({
      appId: config.get('bot.id'),
      appPassword: config.get('bot.secret')
    })

    super(connector, (session) => this.help(session))

    this.set('storage', new BotStorage())
  
    this.use(builder.Middleware.sendTyping(), new MessageSanitizer())

    this.on('conversationUpdate', (message: any) => {
      console.log(message)
    })

    const recognizer = new builder.LuisRecognizer(config.get('bot.model'))
    this.recognizer(recognizer)

Dialog Setup

 this.dialog('/send', [(session, context, next) => {
      const amount = builder.EntityRecognizer.findEntity(context.intent.entities, 'builtin.currency')
      const recipient = builder.EntityRecognizer.findEntity(context.intent.entities, 'recipient')
      const product = builder.EntityRecognizer.findEntity(context.intent.entities, 'product')

      session.send('Sure, I can do that.')
      session.beginDialog('/send/product', product ? product.entity : null)
    }]).triggerAction({
      matches: 'send'
    })

    this.dialog('/send/product', [(session, query, next) => {
      if (query) {
        session.dialogData.productQuery = query
        next()
      } else {
        builder.Prompts.text(session, 'What type of product did you want to send?')
      }
    }, (session, results) => {
      if (results && results.response) {
        session.dialogData.productQuery = results.response
      }

      session.sendTyping()
      ProductService.search(session.dialogData.productQuery).then(products => {
        if (!products.length) {
          session.send('Sorry, I couldn\'t find any products by that name.')
          session.replaceDialog('/send/product')
        }

        const attachments = products.map(product => {
          const image = builder.CardImage.create(session, product.configuration.image)
          const valueLine = `$${product.value.min} - $${product.value.max}`

          const card = new builder.HeroCard(session)
            .title(product.name)
            .images([image])
            .text(product.description)
            .subtitle(valueLine)
            .tap(builder.CardAction.postBack(session, product.id))

          return card
        })

        const message = new builder.Message(session)
          .text('Okay, I found the following products. Please select the one you\'d like to send.')
          .attachments(attachments)
          .attachmentLayout(builder.AttachmentLayout.carousel)

        builder.Prompts.text(session, message)
      }).catch((err: Error) => {
        session.error(err)
      })
    }, (session, response, next) => {
      console.log(response)
    }])
  }

Steps to Reproduce

  1. Create a UniversalBot configured with a LUIS recognizer.
  2. Initiate a LUIS-recognized dialog and then prompt the user for some input using a built-in prompt.
  3. Respond to the prompt and observe that the bot attempts to begin a new dialog as if it is re-recognizing your response rather than resuming the dialog.

Expected Behavior

Resuming the suspended dialog and passing the user response to the next waterfall step.

Actual Results

Bot attempts to start a brand new dialog based on its attempts to match the intent of the user's response.

@dkushner
Copy link
Author

dkushner commented May 1, 2017

For reference, it looks like the guy in #424 was having the same issue as I am on a much older version. Never seems to have been fixed/addressed.

@delight-by
Copy link

That's probably how it is designed to work. As I see you have a recognizer at bot level (you can also have them on library and dialog levels). When the bot handles a response (even during a prompt dialog) it needs to know where to route it and that's when all root recognizers are run. That's also where your triggerAction comes to play - it becomes a root intent handler which can be triggered any time your root recognizer returns "send" intent. And there's a special fallback route named "ActiveDialog", which is just one of the outcomes of the routing process, and only that will route the message to the prompt dialog you've previously started.
Having bot recognizers and triggerActions is what will allow your bot behave in a more flexible way, but it will also make your bot less predictable especially taking NLU into equation. So you really have to find a balance that works for you. I would suggest you:

  • Do not use recognizers at bot level, use them at dialog level (via IntentDialog as root dialog).
  • Use separate LUIS model at root level (or e.g. API.AI model with input context to limit the number of intents you may recognize)

@dkushner
Copy link
Author

dkushner commented May 1, 2017

@delight-by, I recognize only one, very simple intent via the LUIS model. I have tried configuring the recognizer both at the bot level and at the IntentDialog level. Here is the alternate version that suffers from the same issues:

    const recognizer = new builder.LuisRecognizer(config.get('bot.model'))
    this.intents = new builder.IntentDialog({ 
      recognizers: [recognizer],
      recognizeOrder: builder.RecognizeOrder.series
    })

    this.intents.matches('send', '/send')
    this.intents.onDefault((session) => this.help(session))
    
    this.dialog('/', this.intents)
    this.dialog('/send', [(session, context, next) => {
      const amount = builder.EntityRecognizer.findEntity(context.entities, 'builtin.currency')
      const recipient = builder.EntityRecognizer.findEntity(context.entities, 'recipient')
      const product = builder.EntityRecognizer.findEntity(context.entities, 'product')

      session.send('Sure, I can do that.')
      session.beginDialog('/send/product', product ? product.entity : null)
    }, (session) => {
      console.log('hello')
    }])

    this.dialog('/send/product', [(session, query, next) => {
      if (query) {
        session.dialogData.productQuery = query
        next()
      } else {
        builder.Prompts.text(session, 'What type of gift card did you want to send?')
      }
    }, (session, results) => {
      if (results && results.response) {
        session.dialogData.productQuery = results.response
      }

      session.sendTyping()
      ProductService.search(session.dialogData.productQuery).then(products => {
        if (!products.length) {
          session.send('Sorry, I couldn\'t find any products by that name.')
          session.replaceDialog('/send/product')
        }

        const attachments = products.map(product => {
          const image = builder.CardImage.create(session, product.configuration.image)
          const valueLine = `$${product.value.min} - $${product.value.max}`

          const card = new builder.HeroCard(session)
            .title(product.name)
            .images([image])
            .text(product.description)
            .subtitle(valueLine)
            .tap(builder.CardAction.postBack(session, product.id))

          return card
        })

        const message = new builder.Message(session)
          .text('Okay, I found the following products. Please select the one you\'d like to send.')
          .attachments(attachments)
          .attachmentLayout(builder.AttachmentLayout.carousel)

        builder.Prompts.text(session, message)
      }).catch((err: Error) => {
        session.error(err)
      })
    }, (session, response, next) => {
      console.log(response)
    }])

Also, I'm not seeing how this should be intended behaviour. If the recognizer is used to match the original intent, that intent spawns a dialog, then every subsequent dialog/prompt should be in the context of that original dialog. The most basic LUIS example given in the documentation shows setting up an alarm in a similar fashion. The intent is used to initiate the dialog but then they use prompts to gather additional user information. How would this be different? What am I doing wrong?

@delight-by
Copy link

delight-by commented May 1, 2017

every subsequent dialog/prompt should be in the context of that original dialog

That should be true if you're not using recognizers at bot/library levels. I haven't run your code, but visually it looks right. What problem do you have with it now?

Also, have you tried 3.8.0 beta? They completely redesigned Prompts there. But from my experience it has a bug when a prompt doesn't recognize an utterance (e.g. a number prompt and not-a-number response) - it is spamming with a WARNs about not being able to match null intent (but it is just a WARN, otherwise it was working correctly). Install as botbuilder@3.8.0-beta8.

@dkushner
Copy link
Author

dkushner commented May 1, 2017

@delight-by, the problem I have is that when it reaches any of the prompts, instead of the user's response being associated with that prompt it is evaluated as a new intent. It should be passed to the next waterfall step in that dialog as the results parameter.

I'll give 3.8.0 a try.

@dkushner
Copy link
Author

dkushner commented May 1, 2017

@delight-by, same issue persists with 3.8.0

@nwhitmont nwhitmont changed the title Prompt response starts new dialog with LUIS recognizer. [LUIS] Prompt response starts new dialog with LUIS recognizer. May 1, 2017
@nwhitmont nwhitmont added the LUIS label May 5, 2017
@nwhitmont
Copy link
Contributor

A match in the triggerAction will take precedence as every message received by the bot is sent through the routing system. You can customize it, but you can also try using the IntentDialog to isolate some of your flows from it. More info

@dkushner It looks like your question was answered on Stack Overflow - do you require additional assistance?

@Stevenic
Copy link
Contributor

Stevenic commented May 9, 2017

As stated above, trigger actions will trigger a new instance of a dialog anytime the user says something matching a given intent. This is generally desirable but if you don't want this behavior you have a couple of options. You could use an IntentDialog instead of trigger actions as they don't allow interruptions of running tasks. You could also customize your trigger actions by providing an onFindAction handler. Or a simpler approach might be to customize your recognizer to filter out certain intents while a dialog is running.

In 3.8 I added the ability to customize recognizers using new onEnabled and onFilter methods. Here's an example of adding a filter which disables the recognizer anytime a task is running:

var recognizer = new builder.LuisRecognizer('<model>').onEnabled(function (context, callback) {
     var enabled = context.dialogStack().length == 0;
     callback(null, enabled);
});

@nwhitmont nwhitmont added the customer-replied-to Indicates that the team has replied to the issue reported by the customer. Do not delete. label May 19, 2017
@nwhitmont nwhitmont moved this from To Do to Done in Open Issues - Node SDK May 19, 2017
@jdnichollsc
Copy link

@Stevenic what do you think about a function to enable/disable LUIS recognizer?

@vwart
Copy link

vwart commented May 23, 2018

Your desired behaviour can be achieved with "onSelectAction". Have a look here:
https://docs.microsoft.com/en-us/azure/bot-service/nodejs/bot-builder-nodejs-dialog-manage-conversation-flow?view=azure-bot-service-3.0
You can begin a new dialog without clearing the stack this way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
customer-replied-to Indicates that the team has replied to the issue reported by the customer. Do not delete.
Projects
No open projects
Development

No branches or pull requests

6 participants