Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Testing other dialogues #77

Closed
Stikemanley opened this issue May 8, 2018 · 7 comments
Closed

Testing other dialogues #77

Stikemanley opened this issue May 8, 2018 · 7 comments
Labels
question

Comments

@Stikemanley
Copy link

Hello,

We have a fairly complex bot set up with many dialogues, middleware, etc... I'd love a simple way of having a set of questions and answers that i can run through this test harness, but it appears as though every dialogue in the documentation is created inside of the harness. Is it straightforward to have the harness use the full extent of dialogues we've created?

Thanks
@microsoftly
Copy link
Owner

It absolutely can! The tests were simple to demonstrate usage.

If it's possible, I would create a clean instance of the bot for each test or collection of related tests, however this isn't absolutely necessary.

If you want a full e2e test of a dialog, you could run an example like this:
suppose you want to test the restaurant selection of the following dialog
user: Hello
bot: Hi there! What can I help you with?
user: I'd like to book a reservation
bot: awesome!, for what date and time?
user: May 29th at 7pm
bot: Got it! Now what restaurant would you like
user: <RESPONSE GOES HERE>
// bot then verifies restaurant here

you could have a function that builds the conversation to that point and seed the relevant test with that.

Try running some simple tests with your existing bot to get a feel for what it takes to test certain facets of your bot. It should be relatively straightforward.

Lmk if this helps or you need any further insight.

@Stikemanley
Copy link
Author

Stikemanley commented May 11, 2018 via email

@microsoftly
Copy link
Owner

What do you mean by separate the file containing your tests? From what are you separating it?

@Stikemanley
Copy link
Author

Stikemanley commented May 11, 2018 via email

@microsoftly
Copy link
Owner

ahh. Yes. Here's a typescript snippet of a test data provider I wrote for a customer-agent handoff library

const bot = {
    id: 'bot',
    name: 'Bot'
};

const channelId = 'test';
const AGENT_1_USER: IIdentity = {
    id: 'agent1Id',
    name: 'agent1'
};

const AGENT_2_USER: IIdentity = {
    id: 'agent2Id',
    name: 'agent2'
};

const AGENT_1_CONVO_1: IAddress = {
    bot,
    channelId,
    user: AGENT_1_USER,
    conversation: {
        id: 'agent1Convo1'
    }
};

const AGENT_1_CONVO_2: IAddress = {
    bot,
    channelId,
    user: AGENT_1_USER,
    conversation: {
        id: 'agent1Convo2'
    }
};

const AGENT_2_CONVO_1: IAddress = {
    bot,
    channelId,
    user: AGENT_2_USER,
    conversation: {
        id: 'agent2Convo1'
    }
};

const AGENT_2_CONVO_2: IAddress = {
    bot,
    channelId,
    user: AGENT_2_USER,
    conversation: {
        id: 'agent2Convo2'
    }
};

const CUSTOMER_1: IAddress = {
    bot,
    channelId,
    user: {
        id: 'customer1Id',
        name: 'customer1'
    },
    conversation: {
        id: 'customer1ConvoId'
    }
};

const CUSTOMER_2: IAddress = {
    bot,
    channelId,
    user: {
        id: 'customer2Id',
        name: 'customer2'
    },
    conversation: {
        id: 'customer2ConvoId'
    }
};

const CUSTOMER_1_MESSAGE_1 = new Message()
    .text('customer 1 FIRST message')
    .address(CUSTOMER_1)
    .toMessage();

const CUSTOMER_2_MESSAGE_1 = new Message()
    .text('customer 2 FIRST message')
    .address(CUSTOMER_2)
    .toMessage();

const CUSTOMER_1_MESSAGE_2 = new Message()
    .text('customer 1 SECOND message')
    .address(CUSTOMER_1)
    .toMessage();

const CUSTOMER_2_MESSAGE_2 = new Message()
    .text('customer 2 SECOND message')
    .address(CUSTOMER_2)
    .toMessage();

const CUSTOMER_1_MESSAGE_3 = new Message()
    .text('customer 1 THIRD message')
    .address(CUSTOMER_1)
    .toMessage();

const CUSTOMER_2_MESSAGE_3 = new Message()
    .text('customer 2 THIRD message')
    .address(CUSTOMER_2)
    .toMessage();

const AGENT_1_CONVO_1_MESSAGE_1 = new Message()
    .text('agent 1 convo 1 FIRST message')
    .address(AGENT_1_CONVO_1)
    .toMessage();

const AGENT_1_CONVO_2_MESSAGE_1 = new Message()
    .text('agent 1 convo 2 FIRST message')
    .address(AGENT_1_CONVO_2)
    .toMessage();

const AGENT_2_CONVO_1_MESSAGE_1 = new Message()
    .text('agent 2 convo 1 FIRST message')
    .address(AGENT_2_CONVO_1)
    .toMessage();

const AGENT_2_CONVO_2_MESSAGE_1 = new Message()
    .text('agent 2 convo 2 FIRST message')
    .address(AGENT_2_CONVO_2)
    .toMessage();

const AGENT_1_CONVO_1_MESSAGE_2 = new Message()
    .text('agent 1 convo 1 SECOND message')
    .address(AGENT_1_CONVO_1)
    .toMessage();

const AGENT_1_CONVO_2_MESSAGE_2 = new Message()
    .text('agent 1 convo 2 SECOND message')
    .address(AGENT_1_CONVO_2)
    .toMessage();

const AGENT_2_CONVO_1_MESSAGE_2 = new Message()
    .text('agent 2 convo 1 SECOND message')
    .address(AGENT_2_CONVO_1)
    .toMessage();

const AGENT_2_CONVO_2_MESSAGE_2 = new Message()
    .text('agent 2 convo 2 SECOND message')
    .address(AGENT_2_CONVO_2)
    .toMessage();

const AGENT_1_CONVO_1_MESSAGE_3 = new Message()
    .text('agent 1 convo 1 THIRD message')
    .address(AGENT_1_CONVO_1)
    .toMessage();

const AGENT_1_CONVO_2_MESSAGE_3 = new Message()
    .text('agent 1 convo 2 THIRD message')
    .address(AGENT_1_CONVO_2)
    .toMessage();

const AGENT_2_CONVO_1_MESSAGE_3 = new Message()
    .text('agent 2 convo 1 THIRD message')
    .address(AGENT_2_CONVO_1)
    .toMessage();

const AGENT_2_CONVO_2_MESSAGE_3 = new Message()
    .text('agent 2 convo 2 THIRD message')
    .address(AGENT_2_CONVO_2)
    .toMessage();

export function getExpectedReceivedMessage(originalMessage: IMessage, expectedAddress: IAddress): IMessage {
    return Object.assign({}, originalMessage, { address: expectedAddress });
}

// tslint:disable
// I'm too lazy to build the return type for this ...
function getCustomerEventMessages(customerAddress: IAddress) {
// tslint:enable
    return {
        toQueue: new QueueEventMessage(customerAddress),
        toDequeue: new DequeueEventMessage(customerAddress)
    };
}

//tslint:disable
// I'm too lazy to type out the return type
function getAgentEventMessagesForConversation(convoAddress: IAddress) {
//tslint:enable
    return {
        toConnectTo: {
            customer1: new ConnectEventMessage(CUSTOMER_1, convoAddress),
            customer2: new ConnectEventMessage(CUSTOMER_2, convoAddress)
        },
        toDisconnectFrom: {
            customer1: new DisconnectEventMessage(CUSTOMER_1, convoAddress),
            customer2: new DisconnectEventMessage(CUSTOMER_2, convoAddress)
        },
        toWatch: {
            customer1: new WatchEventMessage(CUSTOMER_1, convoAddress),
            customer2: new WatchEventMessage(CUSTOMER_2, convoAddress)
        },
        toUnwatch: {
            customer1: new UnwatchEventMessage(CUSTOMER_1, convoAddress),
            customer2: new UnwatchEventMessage(CUSTOMER_2, convoAddress)
        }
    };
}

export const agent1 = {
    user: AGENT_1_USER,

    convo1: {
        address: AGENT_1_CONVO_1,

        message1: AGENT_1_CONVO_1_MESSAGE_1,
        message2: AGENT_1_CONVO_1_MESSAGE_2,
        message3: AGENT_1_CONVO_1_MESSAGE_3,

        eventMessage: getAgentEventMessagesForConversation(AGENT_1_CONVO_1)
    },

    convo2: {
        address: AGENT_1_CONVO_2,

        message1: AGENT_1_CONVO_2_MESSAGE_1,
        message2: AGENT_1_CONVO_2_MESSAGE_2,
        message3: AGENT_1_CONVO_2_MESSAGE_3,

        eventMessage: getAgentEventMessagesForConversation(AGENT_1_CONVO_2)
    }
};

export const agent2 = {
    user: AGENT_2_USER,

    convo1: {
        address: AGENT_2_CONVO_1,

        message1: AGENT_2_CONVO_1_MESSAGE_1,
        message2: AGENT_2_CONVO_1_MESSAGE_2,
        message3: AGENT_2_CONVO_1_MESSAGE_3,

        eventMessage: getAgentEventMessagesForConversation(AGENT_2_CONVO_1)
    },

    convo2: {
        address: AGENT_2_CONVO_2,

        message1: AGENT_2_CONVO_2_MESSAGE_1,
        message2: AGENT_2_CONVO_2_MESSAGE_2,
        message3: AGENT_2_CONVO_2_MESSAGE_3,

        eventMessage: getAgentEventMessagesForConversation(AGENT_2_CONVO_2)
    }
};

export const customer1 = {
    address: CUSTOMER_1,

    message1: CUSTOMER_1_MESSAGE_1,
    message2: CUSTOMER_1_MESSAGE_2,
    message3: CUSTOMER_1_MESSAGE_3,

    eventMessage: getCustomerEventMessages(CUSTOMER_1)
};

export const customer2 = {
    address: CUSTOMER_2,

    message1: CUSTOMER_2_MESSAGE_1,
    message2: CUSTOMER_2_MESSAGE_2,
    message3: CUSTOMER_2_MESSAGE_3,

    eventMessage: getCustomerEventMessages(CUSTOMER_2)
};

I use it like this:

    it('can handover to agents', async () => {
        await new BotTester(bot, { defaultAddress: TestDataProvider.customer1.address })
            .sendMessageToBotIgnoringResponseOrder(
                new ConnectEventMessage(TestDataProvider.customer1.address, TestDataProvider.agent1.convo1.address))
                .wait(EVENT_DELAY)
            .sendMessageToBot(
                TestDataProvider.agent1.convo1.message1,
                TestDataProvider.getExpectedReceivedMessage(TestDataProvider.agent1.convo1.message1, TestDataProvider.customer1.address))
            .sendMessageToBot(
                TestDataProvider.customer1.message2,
                TestDataProvider.getExpectedReceivedMessage(TestDataProvider.customer1.message2, TestDataProvider.agent1.convo1.address)
            )
            .runTest()
            .then(() => providerSpy.getAllConversations())
            .then((convos: IConversation[]) => {
                convos.forEach((convo: IConversation) => {
                    expect(convo.transcript).not.to.be.empty;
                });
            });
    });

@Stikemanley
Copy link
Author

Stikemanley commented May 11, 2018 via email

@microsoftly
Copy link
Owner

Would you mind formatting? I'm having trouble reading it in it's current layout

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@microsoftly @Stikemanley