Integration test for Microsoft's Bot Framework using Direct Line channel

This repository has a test solution in Visual Studio that does integration test for Microsoft's Bot Framework using the DirectLine channel.

Table of contents

• Working flow
• Project tree
  • Auth
  • Collections
  • Objects
  • Tests
  • Utils
• JSON structure
• Building the JSON
  • Requirements
  • Build the main structure of the JSON
  • Building an entry
  • Filling the steps
    • Sending a message and getting the Activity
    • Receiving a message and getting the Activity
    • Building the assert
    • Not asserting the message
• Contributing
• License

Working flow

Project tree

I've separated the classes by folders as you can see in the following tree.

│
├───Auth
│       DirectLineAuth.cs
│
├───Collections
│       TestEntriesCollection.cs
│
├───Objects
│       TestEntry.cs
│       TestSteps.cs
│
├───Tests
│       DirectLineTests.cs
│       FileTests.cs
│       IntegrationTests.cs
│
└───Utils
        ActivityResponse.cs
        API.cs
        Data.cs
        Globals.cs

Auth

• DirectLineAuth.cs contains the object for the received json in the authorization process.

Collections

• TestEntriesCollection.cs contains the object for the json that contains the information and the entries.

Objects

• TestEntry.cs contains the object for the json that contains the entry to test.
• TestSteps.cs contains the object for the json that contains the step that will be made by the bot.

Tests

• DirectLineTests.cs contains the testing cases for the authetication.
• FileTests.cs contains the testing cases for the file.
• IntegrationTests.cs contains thetesting for the integration test cases.

Utils

• ActivityResponse.cs contains the object for the reveiced json when asking for all the messages in a determined conversation.
• Globals.cs contains the object for loading the objects in the CodeAnalysis evaluator.
• Data.cs contains the string for the cases file path.
• API.cs contains functions for API calls.

JSON structure

The json structure contains all the information necessary for the bot.

• string secret: DirectLine secret
• string directlineGenerateTokenEndpoint: DirectLine endpoint to get the token
• string directlineConversationEndpoint: DirectLine endpoint to get the conversation
• List<TestEntry> entries: Cases
  • string name: Case name
  • string description: Case description
  • bool mute: Enable or disable the case, set this to true to not test the case
  • List<TestSteps> steps: Steps to do in the case
    • Activity request: Request to the bot as Activity
    • Activity response: Request to the bot as Activity
    • bool assert: Assert string to do, have to return a boolean.

{
  "secret": "YourDirectLineChatSecret",
  "directlineGenerateTokenEndpoint": "https://directline.botframework.com/v3/directline/tokens/generate",
  "directlineConversationEndpoint": "https://directline.botframework.com/v3/directline/conversations/",
  "entries": [
    {
      "name": "CaseName",
      "description": "CaseDescription",
      "mute":  false ,
      "steps": [
        {
          "request": {
           /* Request as activity */
          },
          "response": {
            /* Response as activity*/
          },
          "assert": /* Assert as string, have to return boolean */
        },
         {
          "request": {
           /* Request as activity */
          },
          "response": {
             /* Response as activity*/
          },
          "assert": /* Assert as string, have to return boolean */
        },
         {
          "request": {
           /* Request as activity */
          },
          "response": {
             /* Response as activity*/
          },
          "assert": /* Assert as string, have to return boolean */
        }
      ]
    }
  ]
}

Building the JSON

Requirements

1. Microsoft Bot Framework Emulator (V4 PREVIEW)
2. DirectLine Secret
3. DirectLine token generation endpoint, at the moment of this guide is https://directline.botframework.com/v3/directline/tokens/generate.
4. DirectLine conversation creation endpoint, at the moment of this guide is https://directline.botframework.com/v3/directline/conversations.
5. Any text editor to build the JSON.

Build the main structure of the JSON

Here will be the main configuration for the JSON file.

{
  "secret": /* Secret from DirectLine */,
  "directlineGenerateTokenEndpoint": /* DirectLine token endpoint from above */,
  "directlineConversationEndpoint": /* DirectLine conversation token from above */,
  "entries": [
    /* Here will be the cases */
  ]
}

Building an entry

An entry contains some information about it: name, description, mute and steps.

{
    "name": /* Here will be the case name */,
    "description": /* Here will be the case description */,
    "mute":  /* Here will be if the case is enabled or not*/,
    "steps": [
      {
        "request": {
          /* Request as activity */
        },
        "response": {
          /* Response as activity*/
        },
        "assert": /* Assert as string, have to return boolean */
      },
        {
        "request": {
          /* Request as activity */
        },
        "response": {
            /* Response as activity*/
        },
        "assert": /* Assert as string, have to return boolean */
      },
        {
        "request": {
          /* Request as activity */
        },
        "response": {
            /* Response as activity*/
        },
        "assert": /* Assert as string, have to return boolean */
      }
    ]
  }

Filling the steps

In order to fill the steps, we need to use the Microsoft Bot Framework Emulator (V4 PREVIEW), using this application we can easily read all the serialized Activity that we are sending/receiving.

Sending a message and getting the Activity

Pick the message you want, to send and add it to the request.

So far the step looks like this:

{
  "request": {
    "channelData": {
      "clientActivityId": "112983719237123798"
    },
    "entities": [
      {
        "requiresBotState": true,
        "supportsListening": true,
        "supportsTts": true,
        "type": "ClientCapabilities"
      }
    ],
    "from": {
      "id": "default-user",
      "name": "User"
    },
    "id": "7b2c9300-6002131313131317830d951",
    "locale": "es",
    "text": "Hola",
    "textFormat": "plain",
    "timestamp": "2018-05-25T10:00:40.747Z",
    "type": "message"
  },
  "response": {},
  "assert": ""
}

Receiving a message and getting the Activity

Pick the message that you will be comparing and add it to the response.

Message should be the last message you will receive, for example: if you ask hi and the bot returns 3 messages, you have to add the last one

We have the respone now, o far the step looks like this:

{
  "request": {
    "channelData": {
      "clientActivityId": "112983719237123798"
    },
    "entities": [
      {
        "requiresBotState": true,
        "supportsListening": true,
        "supportsTts": true,
        "type": "ClientCapabilities"
      }
    ],
    "from": {
      "id": "default-user",
      "name": "User"
    },
    "id": "7b2c9300-6002131313131317830d951",
    "locale": "es",
    "text": "Hola",
    "textFormat": "plain",
    "timestamp": "2018-05-25T10:00:40.747Z",
    "type": "message"
  },
  "response": {
    "attachments": [],
    "channelId": "emulator",
    "conversation": {
      "id": "74a9db50-600asdasdadb28|livechat"
    },
    "entities": [],
    "from": {
      "id": "ba2e1c30-asdasd54fafccc92",
      "name": "Bot"
    },
    "id": "7be079b0-6002-11e8-b284-ffdb7830d951",
    "localTimestamp": "2018-05-25T11:00:41+01:00",
    "locale": "es",
    "membersAdded": [],
    "membersRemoved": [],
    "reactionsAdded": [],
    "reactionsRemoved": [],
    "recipient": {
      "id": "default-user",
      "role": "user"
    },
    "replyToId": "7b2c93asdasdasddb7830d951",
    "serviceUrl": "https://aaae2b27.ngrok.io",
    "text": "Selecciona una de las siguientes opciones:",
    "timestamp": "2018-05-25T10:00:41.931Z",
    "type": "message"
  },
  "assert": ""
}

Building the assert

In order to make the assert, the string evaluation has to return a boolean, a few examples:

1. "Response.Attachments.Count >= 3"
2. "Response.Text == \"Message\""
3. "Response.Attachments[0].Text == "Hello""

In this case we will be comparing to the text, so our assert value will be:

"assert": Response.Text == \"Selecciona una de las siguientes opciones:\"".

In the end, our step will be like this:

{
  "request": {
    "channelData": {
      "clientActivityId": "112983719237123798"
    },
    "entities": [
      {
        "requiresBotState": true,
        "supportsListening": true,
        "supportsTts": true,
        "type": "ClientCapabilities"
      }
    ],
    "from": {
      "id": "default-user",
      "name": "User"
    },
    "id": "7b2c9300-6002131313131317830d951",
    "locale": "es",
    "text": "Hola",
    "textFormat": "plain",
    "timestamp": "2018-05-25T10:00:40.747Z",
    "type": "message"
  },
  "response": {
    "attachments": [],
    "channelId": "emulator",
    "conversation": {
      "id": "74a9db50-6asdasdasdb28|livechat"
    },
    "entities": [],
    "from": {
      "id": "ba2e1c30-5360-1asdasdasdfafccc92",
      "name": "Bot"
    },
    "id": "7be079b0-60asdasdasddb7830d951",
    "localTimestamp": "2018-05-25T11:00:41+01:00",
    "locale": "es",
    "membersAdded": [],
    "membersRemoved": [],
    "reactionsAdded": [],
    "reactionsRemoved": [],
    "recipient": {
      "id": "default-user",
      "role": "user"
    },
    "replyToId": "7b2c9300-60asdasdasdffdb7830d951",
    "serviceUrl": "https://aaae2b27.ngrok.io",
    "text": "Selecciona una de las siguientes opciones:",
    "timestamp": "2018-05-25T10:00:41.931Z",
    "type": "message"
  },
  "assert": Response.Text == \"Selecciona una de las siguientes opciones:\""
}

So with this example, you can add more steps to the main JSON and make a flow conversation.

Not asserting the message

If you just want to send a message without testing it, all you have to do is to leave empty the assert.

Contributing

As a developer, if you feel like helping, any contribution is welcome.

And as user, if youy have any bug, issue, feature request or question, feel free to open a ticket issue.

License

MIT License

Copyright (c) 2018 Emiliano Montesdeoca del Puerto

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.