Automated Testing for Chatbots
Latest commit d19d726 Sep 4, 2018

Test My Bot


Codeship Status for codeforequity-at/testmybot npm version license

TestMyBot is a test automation library for chatbots. Test cases are recorded by Capture & Replay tools and can be run against your chatbot implementation automatically over and over again. It is meant to be included in your continuous integration pipeline, just as with your unit tests.


Please navigate to our Wiki for documentation.


There are a couple of samples available, showing possible scenarios to use TestMyBot. For all samples, you these commands to run them:

> npm install
> npm run test

To run the console emulator (text input and output), run this command in the sample directory:

> npm install
> npm run emulator

In case you want to see some more output, enable debug logging:

> export DEBUG=testmybot*,botium*

Loading ready-to-use Chatbot Test Cases and Utterances from "Botium Utterances" library


  • Demonstrates usage of the free utterances and chatbot test cases library "Botium Utterances"
  • Loads the typical "Tell me a Joke" test suite and runs it agains a Dialogflow chatbot

Chatbot developed with Microsoft Bot Framework


  • Uses one of the sample chatbots included in the Bot Framework Samples Repository
  • Loads chatbot code from Github automatically
  • Uses "docker" mode to place chatbot in docker container

Chatbot developed with Botkit


  • Wires Botkit with TestMyBot

Loading a Facebook chatbot from Git


  • Uses a calculator sample
  • Loads chatbot code from Github automatically
  • Uses "docker" mode to place chatbot in docker container

(no test cases contained in this sample)

Chatbot developed in Clojure


  • Shows a chatbot developed in another programming language than TestMyBot
  • Uses a chatbot developed with Clojure from the Lemmings incubator
  • Downloads a docker image ("clojure.lein") and placing the chatbot in a docker container
  "botium": {
    "Capabilities": {
      "PROJECTNAME": "testmybot-sample-clojure",
      "FACEBOOK_API": true,
      "FACEBOOK_WEBHOOK_PATH": "webhook",
      "CLEANUPTEMPDIR": false,
	    "DOCKERIMAGE": "clojure:lein",
      "STARTCMD": "lein ring server-headless"
    "Sources": {
      "GITURL": ""

Sample Facebook Chatbot (Node.js)


  • Shows how to include TestMyBot into your build pipeline for continuous testing
  • docker container

Connecting to Chatbot running in Facebook Messenger


  • Can be used to automate testing of a chatbot already running in facebook messenger
  • Usefull for regression testing, not for continous testing (takes to much time to run)
  • See sample directory for instructions

Using Mocha


  • Mocha can be used instead of Jasmine as well
  • docker container

Chatbot developed in PHP


  • Chatbot developed in PHP
  • See sample directory for instructions

Slack Chatbot


  • Sample how to test a Slack chatbot

IBM Watson Chatbot


  • Connecting to Watson conversation service to run your convos
  • Useful for regression testing
  • Workspace can be copied automatically before running the test cases (to not pollute your workspace)

Voice recognition and speech synthesis


  • Uses the Web speech api to run a conversation with Alexa
  • Requires a real browser supporting the Web Speech API

Custom REST/HTTP interface and using Excel test cases


  • Contacting a custom REST/HTTP interface, as usual for chatbots running on websites
  • Test cases are extracted from an Excel sheet instead of text files