Skip to content

Write your own test case

Jump to bottom
mgmschoene edited this page Nov 10, 2020 · 23 revisions

Follow these steps to write your own test case and contribute it to CanITrust.in:

Preparation

Docker host

To be able to locally run the test cases and make sure your code runs, you will need a local docker and docker-compose installation. To test whether you have docker installed, enter

docker --version
docker-compose --version

If one of these commands does not return a version number, please follow one of these guides to install docker and docker-compose:

Look at the code

Clone the repo

git clone git@github.com:canitrust/backend.git
cd backend

Test case structure

Our test cases are located in the testcases directory.

cd driver/testcases

It makes sense to first check out the structure of an existing test case, e.g.

cat case6.py

Each test case needs to import and subclass the base test case and have an __init__ method. Class name and testCaseNum correspond to the number of the test case:

from testcases.testCase import TestCase
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from helper import Logger
logger = Logger(__name__).logger

class Case6(TestCase):

    def __init__(self):
        TestCase.__init__(self)
        self.testCaseNum = 6

The actual test is defined in the method executeTest() using Selenium webdriver for Python:

    def executeTest(self, webDriver):
        webDriver.get("https://ssl.test-canitrust.com")
        WebDriverWait(webDriver, 10).until(EC.presence_of_element_located((By.TAG_NAME, 'body')))
        cookies = webDriver.get_cookies()
        webDriver.close()
        self.data = { 'data_cookies': cookies }
        return 1

This example test case performs the following steps:

  1. Call the URL https://ssl.test-canitrust.com.
  2. Wait until the page is loaded.
  3. Get the cookies set by the page.
  4. Close the webdriver.
  5. Pass the cookies as test results to the evaluation step by setting them to self.data.

In the final step, we convert the test data into test result in the method evaluate():

    def evaluate(self):
        result = 0
        for cookie in self.data['data_cookies']:
            if cookie['name'] == 'cookie0':
                if cookie['value'] == 'value1':
                    result = 2
                else:
                    result = 3
        self.result = result

Here, the following steps are happening:

  1. Search for a cookie named 'cookie0' in the test data. If no such cookie exists, the test result is 0.
  2. Check whether its value is 'value1'. if yes, the test result is 2. Otherwise, the test result is 3.

The test result numbers assigned will later decide in which colour the test result will be displayed in the frontend.

Run the example locally

To spin up the test environment and take the example code for a run locally, do the following:

cd ..
./driver.sh runlocal -t 6

This will build all the necessary docker containers and start them. Note: Your ports 8080 and 8081 must be unused (compare docker-compose.local.yml). You should see some output similar to this:

Creating network "backend_default" with the default driver
2019-12-10 06:43:33,307 - __main__ - INFO - Run Test Cases
2019-12-10 06:43:33,316 - __main__ - INFO - Environment: Local
2019-12-10 06:43:33,317 - __main__ - INFO - List test cases:['6']
2019-12-10 06:43:33,318 - __main__ - INFO - AMOUNT_LOCAL_TESTS:1
2019-12-10 06:43:33,318 - __main__ - INFO - LOCAL_TESTS:['6']
2019-12-10 06:43:33,322 - __main__ - DEBUG - Wait for dns_server and test_app containers up...
[...]
2019-12-10 06:43:42,970 - __main__ - INFO - Local tests running OK: ['6']
2019-12-10 06:43:42,971 - __main__ - INFO - Local tests running Fail: []
2019-12-10 06:43:42,972 - __main__ - INFO - (*) TEST SUMMARY
+--------+---------------+---------+---------+--------+----------------------------------------------------------------+
| Case # |    Browser    | Version | Elapsed | Result |                          Data                                  |
+--------+---------------+---------+---------+--------+----------------------------------------------------------------+
|   6    | Firefox/Gecko |  latest |   0.8   |   3    | {'data_cookies': [{'name': 'cookie1', 'value': 'value1',...}]} |
+--------+---------------+---------+---------+--------+----------------------------------------------------------------+
Driver DONE

Congratulations, you ran your first local test.

Write your own test case

Preparation for your own test case

Create a new test case file:

cd driver/testcases
touch case23.py

Then open the file map.json and give your test case a unique number:

...
  "23": {
    "modulename": "case23",
    "filename": "case23.py",
    "isLive": false
  },
...

Test case code

Now let's write some code! Open your test case file in your favourite editor:

vim case23.py

and start with a very basic test case:

from testcases.testCase import TestCase
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from helper import Logger
logger = Logger(__name__).logger

class Case23(TestCase):

    def __init__(self):
        TestCase.__init__(self)
        self.testCaseNum = 23

    def executeTest(self, webDriver):
        """ Definition of a testcase
            Test result MUST be set to self.data
        """
        webDriver.get("http://plain.test-canitrust.com")
        WebDriverWait(webDriver, 10).until(EC.presence_of_element_located((By.TAG_NAME, 'body')))
        webDriver.close()
        self.data = { 'no_data': 'some_data' }
        return 1

    def evaluate(self):
        self.result = 1

This example test case actually tests nothing. It just loads a page, wait until it has finished loading and then return some non-sense data. However, if it got executed successfully you should see the following output:

./driver.sh runlocal -t 23

[...]
+--------+---------------+---------+---------+--------+--------------------------+
| Case # |    Browser    | Version | Elapsed | Result |           Data           |
+--------+---------------+---------+---------+--------+--------------------------+
|   23   | Firefox/Gecko |  latest |   0.9   |   1    | {'no_data': 'some_data'} |
+--------+---------------+---------+---------+--------+--------------------------+

Our own test host

As you saw in the above example, we used an already existing test host plain.test-canitrust.com. For most test cases though, we need our own test host. Let's create one! Open the Apache configuration:

vim test_app/src/config/httpd-canitrust.conf

and add our own host:

[...]
<VirtualHost *:80>
    ServerName demo.test-canitrust.com:80
    DocumentRoot "/usr/local/apache2/htdocs"
</VirtualHost>
[...]

As you can see, we added a new host called demo.test-canitrust.com and pointed the document root to a specific folder which contains HTML files and other resources. Make sure to always use the test-canitrust.com domain or otherwise our included DNS server will not know where to point the selenium driver. If required, you can add new hosts to the DNS server.

Custom HTML pages

Speaking about HTML: Let's create a new HTML file for our host:

vim test_app/src/http/demo.html

and add the following code:

<html>
  <head>
    <title>Demo</title>
  </head>
  <body>
    <h1>Demo H1</h1>
  </body>
</html>

From now on, our test server will serve this HTML file under http://demo.test-canitrust.com/demo.html.

You are also create PHP pages, for example to reflect parameters or set response headers dynamically.

Fixing the test case

Now that we have a fancy demo host with some content, we can update our testcase to be more meaningful:

from testcases.testCase import TestCase
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

class Case23(TestCase):

    def __init__(self):
        TestCase.__init__(self)
        self.testCaseNum = 23

    def executeTest(self, webDriver):
        """ Definition of a testcase
            Test result MUST be set to self.data
        """
        webDriver.get("http://demo.test-canitrust.com/demo.html")
        WebDriverWait(webDriver, 10).until(EC.presence_of_element_located((By.TAG_NAME, 'body')))

        content = webDriver.find_element_by_tag_name("h1").text
        self.data = { 'content': content }

        webDriver.close()
        return 1

    def evaluate(self):
        if self.data['content'] == 'Demo H1':
          result = 1
        else:
          result = 0

        self.result = result

Here, a few things happen:

  1. In the executeTest() method, we now load our new HTML file on our new host (http://demo.test-canitrust.com/demo.html) and wait for its body to load.
  2. We use selenium to find the <h1> tag in the body and store its content for later inspection.
  3. in the evaluate() method, we then check if the content is as expected and return result=1 in that case or result=0 otherwise.

Now we can run the testcase again:

./driver.sh runlocal -t 23

[...]
+--------+---------------+---------+---------+--------+------------------------+
| Case # |    Browser    | Version | Elapsed | Result |          Data          |
+--------+---------------+---------+---------+--------+------------------------+
|   23   | Firefox/Gecko |  latest |   0.8   |   1    | {'content': 'Demo H1'} |
+--------+---------------+---------+---------+--------+------------------------+

Et voila, your first complete test case! 🎊🎊🎊

Pull request

With just changing 2 files and creating 2 new ones, you can wrap a nice PR for us:

git add --all                                           
git status

[...]
Changes to be committed:
        modified:   driver/config/map.json
        new file:   driver/testcases/case23.py
        modified:   test_app/src/config/httpd-canitrust.conf
        new file:   test_app/src/http/demo.html

We will then review it, test it against Browserstack and if the result is good, publish it on canitrust.in!

Clone this wiki locally