Problem: No clear way to combine Adaptive Cards with prompts in dialogs.

[JonathanFingold]

Setup:

• Create a waterfall dialog.
• For one of the steps present the user with a choice via an adaptive card.

Issue:

• There's no way to present that choice as a prompt that includes a validation argument.

Discussion:

The user can use the adaptive card to send a response, or just type in a value. There is no clear way to handle this with existing prompts with writing a new one from the base Prompt class.

[cocoke]

Any update of a possible solution ?
I'm in the same situation.

[JonathanFingold]

This is my current theory, but I don't know which direction the team is going in with this...

It seems like the intention for combining adaptive cards with dialogs is to use the card to collect information that you can use to start the dialog, by passing in values in the call to the dialog context's Begin method. In the first step of the dialog, you would check for any input values and...discard any that don't validate. (There is probably a more elegant way to handle this.) You have the same issue with triggering a dialog from a LUIS result that can return entities.

For use of an Adaptive card (or LUIS) from within a dialog, you could add a "sub-dialog" via a dialog container to your dialog set, that you could pass arguments into and use as one of the steps in the "parent" dialog.

[yochay]

The way Adaptive Cards show in a dialog can lead to undesired results. The user can ignore the adaptive card and continue the conversation. At any given time the user can return in history and click on buttons in the card. The card message is 'out of sequence' sync with the rest of the dialog.

@JonathanFingold , as you identified, currently, there is no to 'force' the user to click buttons as the adaptive card have no way to ensure modality. We can think of a new prompt, called Adaptive card prompt that wait for user input from the card. We'll look into this after our current release.

[tusharvaja]

@yochay - do you know when we can have something like apatite card prompt part of v4? We were looking for similar solution and wanted to check the timeline before implementing effort on something custom.

[v-kydela]

Using a ChoicePrompt with PromptAsync will allow you to specify a list of choices that are auto-validated along with a card that displays the choices as buttons. I've implemented such a solution here. It uses a Hero Card which is what V3 automatically generates, but an Adaptive Card can be used the same way.

[tusharvaja]

thanks @v-kydela for reply, but i am getting and exception on line var dc = await Dialogs.CreateContextAsync(turnContext); while returning adaptive card with choice prompt with many buttons.

Error is like: Error reading JArray from JsonReader. Current JsonReader item is not an array: StartObject. Path 'DialogState.DialogStack.$values[0].State.dialogs.DialogStack.$values[0].State.options.Prompt.attachments.$values[0].content.body'.

i feel it is related to microsoft/AdaptiveCards#2148 but not sure. any solutions?

[v-kydela]

I think I see now that I was wrong about Adaptive Cards being able to replace Hero Cards in this situation. Adaptive Cards only have three possible card actions and don't have an equivalent of imBack. An AdaptiveSubmitAction is the only way for an Adaptive Card to send a message to a bot, and that message will have an empty Text property so a prompt won't be able to respond to it. For now I think you'll need to stick to Hero Cards if you want to user cards in prompts.

Incorrect information corrected below

[FRANZKAFKA13]

This documentation (https://docs.microsoft.com/en-us/healthbot/scenario-authoring/adding-cards#getting-data-from-forms) talks about using adaptive cards in prompts to gather data from forms. Unfortunately, they use some kind of GUI and I can't seem to figure out how to implement the card in a prompt without getting an error message. I use Node. Also, the JSON.parse command the documentation uses doesn't work for me.

[v-kydela]

@FRANZKAFKA13 - While Adaptive Cards are great for gathering information and sending it to the bot, the information must be retrieved from the activity's value property and prompts are expecting replies to only come in the form of an activity's text property

[EricDahlvang]

@FRANZKAFKA13 The documentation page you've shared is in reference to authoring adaptive cards in the HealthBot scenario authoring tool. This is different from a bot built using the Bot Builder sdk alone.

More about Microsoft Health Bot Service can be found here: https://docs.microsoft.com/en-us/healthbot/index

[v-kydela]

@tusharvaja
@FRANZKAFKA13
I've changed my mind again! It looks like there's actually a trick you can use to have an Adaptive Card's submit action work as an imBack. All you have to do is set the submit action's data property to a string instead of an object. However, if you want to do this in a prompt and you're using the AdaptiveCards NuGet package instead of just reading card from a JSON file or something, you'll need another trick. There seems to be a bug in the AdaptiveCards package that throws an error when the card is deserialized as part of the dialog state, so you have to make sure you don't set an AdaptiveCard object as the attachment's content property. To convert the Adaptive Card to a normal object you can use the old serialize/deserialize trick: Content = JsonConvert.DeserializeObject(JsonConvert.SerializeObject(card))

[v-kydela]

I've just discovered that the first trick actually isn't necessary. As long as you can get past the JSON error, you can have anything you want in the Data property and an Adaptive Card can still work in a prompt. Choice prompts are smart enough to look inside the activity's Value property to see if the choice is present in there if the Text property is empty.

Incorrect information corrected below

[v-kydela]

Here's an example of how you might use an Adaptive Card with a prompt:

private async Task<DialogTurnResult> PromptWithAdaptiveCardAsync(WaterfallStepContext stepContext, CancellationToken token)
{
	var choices = new[] { "One", "Two", "Three" };

	var card = new AdaptiveCard
	{
		Version = new AdaptiveSchemaVersion(1, 0),
		Body = { new AdaptiveTextBlock("Make your choice") },
		Actions = choices.Select(choice => new AdaptiveSubmitAction
		{
			Title = choice,
			//Data = choice,       // For imBack behavior
			Data = new { choice }, // For postBack behavior
		}).ToList<AdaptiveAction>(),
	};

	return await stepContext.PromptAsync(
		CHOICEPROMPT,
		new PromptOptions
		{
			Choices = ChoiceFactory.ToChoices(choices),
			Prompt = (Activity)MessageFactory.Attachment(new Attachment
			{
				ContentType = AdaptiveCard.ContentType,
				Content = JsonConvert.DeserializeObject(JsonConvert.SerializeObject(card)),
			}),
		},
		token);
}

[Unders0n]

any updates on that? what is the recommended way to make adaptive card as part of the flow in waterfall?
Notes about modality seems a bit strange 'cos as i get it the only way to show comboboxes is to use adaptive cards now (correct me if wrong), so in simple case of asking user for few inputs from comboboxes in a row is exactly use waterfall and use adaptive cards.
@v-kydela i've managed to use your example, but still can't get how to usse value from combobox, not separate buttons, have you found solution to that?

[v-kydela]

Once again I need to correct myself (every comment a new revelation). Choice prompts are not "smart enough to look inside the activity's Value property" on their own. It turns out you can change an incoming activity's Text property before you process it (this won't affect what shows up in the chat history because the channel handles that before the activity ever reaches your bot). I actually had written my own code to manually set the activity's Text property and then had forgotten about it!

@Unders0n - When you say combobox I'm guessing you mean an adaptive choice set. Here's a method that can set your activity's Text property based on what the user selected in the choice set:

private async Task HandleSubmitActionAsync(ITurnContext turnContext)
{
    dynamic value = turnContext.Activity.Value;
    string text = value["combo"]; // The property will be named after your choice set's ID
    text = string.IsNullOrEmpty(text) ? "." : text; // In case the input is empty
    turnContext.Activity.Text = text;
    var dc = await Dialogs.CreateContextAsync(turnContext);
    var result = await dc.ContinueDialogAsync();
}

This could be called from OnTurnAsync once you've determined that the activity has no text:

var activity = turnContext.Activity;

if (activity.Type == ActivityTypes.Message)
{
    if (string.IsNullOrEmpty(activity.Text))
    {
        await HandleSubmitActionAsync(turnContext);
    }
}

[Unders0n]

@v-kydela thanks for your clarification, i've managed it to work though have an issue:
1)when i'm doin my card from .json and using this code to create it:

var cardAttachment = CreateAdaptiveCardAttachment(@".\Dialogs\Main\Resources\SelectGroupCard.json");

return await stepcontext.PromptAsync("SelectGroupCardDialog",
	new PromptOptions
	{
		Prompt = (Activity)MessageFactory.Attachment(new Attachment
		{
			ContentType = AdaptiveCard.ContentType,
			Content = cardAttachment.Content
		}),
	},
	cancellationtoken);

i'm being able to set Text from value of selected dropdown as from your example but it looping thought this step of waterfall and not goin further. I've checked your example and it worked correct only if i leave
Choices = ChoiceFactory.ToChoices(choices),
, if i take it away it also looping. And i can't get how you managed this Choises property, we already have options in our adaptive card, can we make it work without duplicating options?
UPD: Tried my solution, manually added choices, so it only working when value that we clicked on card found in choices array. Is it the onnly way its working?

[Unders0n]

@EricDahlvang solutions mentioned in this topic is definitely looking like hacky workarounds, is Microsoft planning to change design on that and give smth like modal adaptive cards with easy and correct way to handle events? Thank you.

[v-kydela]

@Unders0n - The choice prompt's Choices property is needed in order to validate the result. In the post I linked to earlier I explained that you can set the style of the choice prompt like this to make sure the choices are only shown in the card and not as text: AddDialog(new ChoicePrompt("SelectGroupCardDialog") { Style = ListStyle.None });

I've also made a feature request to bring back cards in prompts and it's been triaged for 4.3. Adaptive cards will never be modal, though. Look into suggested actions because they're meant to serve the purpose of a one-time set of choices. Or if you really want to prevent reuse of cards you can use something like this technique (was written for V3 but can be easily adapted for V4): https://stackoverflow.com/questions/51701003/microsoft-bot-framework-webchat-disable-adaptivecards-submit-buttons-of-previou/51774589#51774589

[Unders0n]

@v-kydela thanks for updates. ListStyle.None is useful for this case.
as for suggested actions i think they not suits my scenario - i need for user to select one of many options (like 10-20) and we can't use luis or some search here cos we need to show options.
Regarding this workaround i've managed to change example to work with dropdown list .
But i think its nontrivial to use .json cards, cos its kinda complicated to parse it and create respective choises (or im incorrect here? gona check tomorrow)

[v-kydela]

@Unders0n - While it may be more difficult to extract the choices using a JSON card than if you generated the card in C#, LINQ's Select method can still make easy work of it. Once you've parsed the JSON into a .NET object you can grab a list of choices from the list of actions in a sort of reverse way from how you've seen me convert the list of choices into a list of actions.

[Unders0n]

I've faced an issue that i can't pass Value along with Title, when i'm creating stepcontext.PromptAsync , Choices in PromptOptions accepts only list of strings. Any solutions to that?
Only workaround i can think of is store result list with values before showing and then on result map result back to value of that collection.

[v-kydela]

@Unders0n - I'm not entirely sure what you're asking, but would you mind posting a question on StackOverflow? This thread is getting sort of long

[Unders0n]

@v-kydela sure, created a question on SO: https://stackoverflow.com/questions/54094943/botbuilder-v-4-dynamic-adaptive-card-with-dropdown-and-capturing-values-on-prom

[gagana-hg]

@v-kydela I followed the above method to combine adaptive card with prompts in waterfall steps.
But adaptive card is re-prompting again once submit action is performed.
Also I would like to know is it possible to go back to previous step if a button is clicked in adaptive card

AddDialog(new ChoicePrompt("AdaptiveCardPrompt") { Style = ListStyle.None });

Calling adaptive card(.json).

private  async Task<DialogTurnResult> GetInfoAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
    var cardAttachment = new Attachment();
    cardAttachment = CreateAdaptiveCardAttachment();

    return await stepContext.PromptAsync("AdaptiveCardPrompt",
    new PromptOptions
    {
            Prompt = (Activity)MessageFactory.Attachment(new Attachment
            {
                ContentType = AdaptiveCard.ContentType,
                Content = cardAttachment.Content
            }),
     }, cancellationToken);
}

The below code will help us to move to previous step. But how do we implement the same with adaptive card.
stepContext.ActiveDialog.State["stepIndex"] =(int)stepContext.ActiveDialog.State["stepIndex"] - 2;

Adaptive card looks like this

I have raised this issue even in stack overflow. https://stackoverflow.com/questions/56180596/i-am-using-bot-framework-v4-3-i-want-to-retrieve-adaptive-card-submit-values

Kindly help me in solving this issue.

[v-kydela]

@gagana-hg - I don't see how a ChoicePrompt is appropriate if you're trying to prompt using the Adaptive Card's input fields. You'll want to use a TextPrompt and then manually serialize the activity's value into the Text property before it reaches the dialog, or you'll want to create your own prompt class that extends ActivityPrompt. We're actually working on a special AdaptiveCardPrompt right now in case you want to wait.

[JonathanFingold]

One problem with cards is that on some channels users can interact with them again, out-of-sync with the current turn. I don't see any obvious way to include an identifier in the activity payload for a card action, but that would be my first inclination. You could then use that identifier to determine which turn the activity should be associated with.

[gagana-hg]

@v-kydela I tried with TextPrompt It is working fine. Thank you. And when can we expect AdaptiveCardPrompt?

[v-kydela]

@gagana-hg - We have no current timeline or estimate for it, but as a customer you should feel free to voice your interest in this, perhaps on this thread: microsoft/botframework-sdk#5396

[mutanttech]

I also require something like AdaptiveCardPrompt for a requirement.

[droidriz]

Any updates on this?

[nikheelneelay]

Hi Team, any updates on this? Or, any workarounds? Currently we are using a custom base prompt class to achieve this. It works fine in all scenarios, except for validations.

`using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Builder.Dialogs.Choices;
using Microsoft.Bot.Schema;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;

namespace CSITBot
{
//Reference: Prompt.cs
///

/// Basic configuration options supported by all prompts.
///

/// The type of the .
public abstract class CustomPromptBase : Dialog
{
private const string PersistedOptions = "options";
private const string PersistedState = "state";

    private readonly PromptValidator<T> _validator;

    public CustomPromptBase(string dialogId, PromptValidator<T> validator = null)
        : base(dialogId)
    {
        _validator = validator;
    }

    public override async Task<DialogTurnResult> BeginDialogAsync(DialogContext dc, object options, CancellationToken cancellationToken = default(CancellationToken))
    {
        if (dc == null)
        {
            throw new ArgumentNullException(nameof(dc));
        }

        if (!(options is PromptOptions))
        {
            throw new ArgumentOutOfRangeException(nameof(options), "Prompt options are required for Prompt dialogs");
        }

        // Ensure prompts have input hint set
        var opt = (PromptOptions)options;
        if (opt.Prompt != null && string.IsNullOrEmpty(opt.Prompt.InputHint))
        {
            opt.Prompt.InputHint = InputHints.ExpectingInput;
        }

        if (opt.RetryPrompt != null && string.IsNullOrEmpty(opt.RetryPrompt.InputHint))
        {
            opt.RetryPrompt.InputHint = InputHints.ExpectingInput;
        }

        // Initialize prompt state
        var state = dc.ActiveDialog.State;
        state[PersistedOptions] = opt;
        state[PersistedState] = new Dictionary<string, object>();

        // Send initial prompt
        await OnPromptAsync(dc.Context, (IDictionary<string, object>)state[PersistedState], (PromptOptions)state[PersistedOptions], false, cancellationToken).ConfigureAwait(false);

        // Customization starts here for AdaptiveCard Response:
        /* Reason for removing the adaptive card attachments after prompting it to user,
         * from the stat as there is no implicit support for adaptive card attachments.
         * keeping the attachment will cause an exception : Newtonsoft.Json.JsonReaderException: Error reading JArray from JsonReader. Current JsonReader item is not an array: StartObject. Path ‘[‘BotAccessors.DialogState’].DialogStack.$values[0].State.options.Prompt.attachments.$values[0].content.body’.
         */
        var option = state[PersistedOptions] as PromptOptions;
        option.Prompt.Attachments = null;
        /* Customization ends here */

        return Dialog.EndOfTurn;
    }

    public override async Task<DialogTurnResult> ContinueDialogAsync(DialogContext dc, CancellationToken cancellationToken = default(CancellationToken))
    {
        if (dc == null)
        {
            throw new ArgumentNullException(nameof(dc));
        }

        // Don't do anything for non-message activities
        if (dc.Context.Activity.Type != ActivityTypes.Message)
        {
            return Dialog.EndOfTurn;
        }

        // Perform base recognition
        var instance = dc.ActiveDialog;
        var state = (IDictionary<string, object>)instance.State[PersistedState];
        var options = (PromptOptions)instance.State[PersistedOptions];

        var recognized = await OnRecognizeAsync(dc.Context, state, options, cancellationToken).ConfigureAwait(false);

        // Validate the return value
        var isValid = false;
        if (_validator != null)
        {
        }
        else if (recognized.Succeeded)
        {
            isValid = true;
        }

        // Return recognized value or re-prompt
        if (isValid)
        {
            return await dc.EndDialogAsync(recognized.Value).ConfigureAwait(false);
        }
        else
        {
            if (!dc.Context.Responded)
            {
                await OnPromptAsync(dc.Context, state, options, true).ConfigureAwait(false);
            }

            return Dialog.EndOfTurn;
        }
    }

    public override async Task<DialogTurnResult> ResumeDialogAsync(DialogContext dc, DialogReason reason, object result = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        // Prompts are typically leaf nodes on the stack but the dev is free to push other dialogs
        // on top of the stack which will result in the prompt receiving an unexpected call to
        // dialogResume() when the pushed on dialog ends.
        // To avoid the prompt prematurely ending we need to implement this method and
        // simply re-prompt the user.
        await RepromptDialogAsync(dc.Context, dc.ActiveDialog).ConfigureAwait(false);
        return Dialog.EndOfTurn;
    }

    public override async Task RepromptDialogAsync(ITurnContext turnContext, DialogInstance instance, CancellationToken cancellationToken = default(CancellationToken))
    {
        var state = (IDictionary<string, object>)instance.State[PersistedState];
        var options = (PromptOptions)instance.State[PersistedOptions];
        await OnPromptAsync(turnContext, state, options, false).ConfigureAwait(false);
    }

    protected abstract Task OnPromptAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, bool isRetry, CancellationToken cancellationToken = default(CancellationToken));

    protected abstract Task<PromptRecognizerResult<T>> OnRecognizeAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, CancellationToken cancellationToken = default(CancellationToken));

    protected IMessageActivity AppendChoices(IMessageActivity prompt, string channelId, IList<Choice> choices, ListStyle style, ChoiceFactoryOptions options = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        // Get base prompt text (if any)
        var text = prompt != null && !string.IsNullOrEmpty(prompt.Text) ? prompt.Text : string.Empty;

        // Create temporary msg
        IMessageActivity msg;
        switch (style)
        {
            case ListStyle.Inline:
                msg = ChoiceFactory.Inline(choices, text, null, options);
                break;

            case ListStyle.List:
                msg = ChoiceFactory.List(choices, text, null, options);
                break;

            case ListStyle.SuggestedAction:
                msg = ChoiceFactory.SuggestedAction(choices, text);
                break;

            case ListStyle.None:
                msg = Activity.CreateMessageActivity();
                msg.Text = text;
                break;

            default:
                msg = ChoiceFactory.ForChannel(channelId, choices, text, null, options);
                break;
        }

        // Update prompt with text and actions
        if (prompt != null)
        {
            // clone the prompt the set in the options (note ActivityEx has Properties so this is the safest mechanism)
            prompt = JsonConvert.DeserializeObject<Activity>(JsonConvert.SerializeObject(prompt));

            prompt.Text = msg.Text;
            if (msg.SuggestedActions != null && msg.SuggestedActions.Actions != null && msg.SuggestedActions.Actions.Count > 0)
            {
                prompt.SuggestedActions = msg.SuggestedActions;
            }

            return prompt;
        }
        else
        {
            msg.InputHint = InputHints.ExpectingInput;
            return msg;
        }
    }
}

}`

`using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Schema;
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;

namespace CSITBot
{
//Reference: TextPrompt.cs
public class CustomAdaptivePrompt : CustomPromptBase
{
public CustomAdaptivePrompt(string dialogId, PromptValidator validator = null)
: base(dialogId, validator)
{
}

    protected async override Task OnPromptAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, bool isRetry, CancellationToken cancellationToken = default(CancellationToken))
    {
        if (turnContext == null)
        {
            throw new ArgumentNullException(nameof(turnContext));
        }

        if (options == null)
        {
            throw new ArgumentNullException(nameof(options));
        }

        if (isRetry && options.RetryPrompt != null)
        {
            await turnContext.SendActivityAsync(options.RetryPrompt, cancellationToken).ConfigureAwait(false);
        }
        else if (options.Prompt != null)
        {
            await turnContext.SendActivityAsync(options.Prompt, cancellationToken).ConfigureAwait(false);
        }
    }

    protected override Task<PromptRecognizerResult<string>> OnRecognizeAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, CancellationToken cancellationToken = default(CancellationToken))
    {
        if (turnContext == null)
        {
            throw new ArgumentNullException(nameof(turnContext));
        }

        var result = new PromptRecognizerResult<string>();
        if (turnContext.Activity.Type == ActivityTypes.Message)
        {
            var message = turnContext.Activity.AsMessageActivity();
            if (!string.IsNullOrEmpty(message.Text))
            {
                result.Succeeded = true;
                result.Value = message.Text;
            }
            /*Add handling for Value from adaptive card*/
            else if (message.Value != null)
            {
                result.Succeeded = true;
                result.Value = message.Value.ToString();
            }
        }

        return Task.FromResult(result);
    }
}

}`

Usage -->
public async Task<DialogTurnResult> CreateAdaptiveReply(WaterfallStepContext stepContext, CancellationToken cancellationToken, string cardName, string title = "") { var reply = stepContext.Context.Activity.CreateReply(); reply.Attachments = new List<Attachment> { new Attachment { Content = GetAdaptiveCardFromJson(cardName,title), ContentType = AdaptiveCard.ContentType, }, }; return await stepContext.PromptAsync( nameof(CustomAdaptivePrompt), new PromptOptions { Prompt = reply, }, cancellationToken).ConfigureAwait(true); }

[gabog]

Closing as part of R10 triage. The community now has an adaptive card prompt: https://github.com/BotBuilderCommunity/botbuilder-community-dotnet/tree/develop/libraries/Bot.Builder.Community.Dialogs.Prompts