Test Case Creation Guide

Introduction

Test case development in Iris is meant to be straightforward and simple. With some basic Python skills and familiarizing yourself with the Iris test APIs, you can be automating test cases in short order. There are plenty of resources for learning the basics of Python. We won't cover that here. This document will help you find the internal Iris resources, explain best practices and describe pitfalls to avoid. By the end of this guide, you should have a good understanding of how to create solid automated test cases with Iris.

---

Iris Test Structure

Let's take a look at the basic structure of an Iris test case. We'll go into the details of each later.

• Software License
• Import Statements
• Base Test Class (which includes):
  • Initialization function
  • Setup Function (optional)
  • Run Function
  • Cleanup Function (optional)

---

Software License

If you write code for any project, there is usually a software license required. Mozilla code requires:

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this file,
# You can obtain one at http://mozilla.org/MPL/2.0/.

Import Statements

The import statements give the test case access to the modules needed to perform actions in a test case. The following import will generally give each test what it needs to work.

from iris.test_case import *

As you create more complex test cases, you may find the need to import additional modules. If you do, those import statements should be alphabetized per PEP8 guidelines.

The Base Test Class

The Iris test runner identifies a test by its class creation. Everything the test runner needs to know will be a member of this class.

class Test(BaseTest):

The Initialization Function

Information provided in the initialization function is used to uniquely identify a test case and determine if and how the test cases should be run. The example below shows initialization for a Firefox test case.

def __init__(self):
    BaseTest.__init__(self)
    self.meta = '\'Open Download Folder\' opens the folder which Firefox is set to save its downloads.'
    self.test_case_id = '99480'
    self.test_suite_id = '1827' 
    self.locales = ['en-US']
    self.exclude = Platform.LINUX
    self.blocked_by = '1513494'

• self.meta is the summary of the TestRail test case. (required)
• self.test_case_id is the TestRail test case ID number. (required)
• self.test_suite_id is the TestRail test suite ID number the test case belongs to. (required)
• self.locales is a list of locales the test case is compatible with. (required)
• self.exclude is a list of platforms the test should not be run on (optional)
• self.blocked_by is a bug or issue number the test is blocked by. (optional)

The Setup Function

If you must use a non-default profile and/or make changes to application preferences prior to the test being run, this is the place to do so. Otherwise, you may leave this out of your test. For example:

def setup(self):
    """
    This overrides the setup method in the BaseTest class, so that it can use a brand new profile and
    alters a few Firefox preferences. 
    """
    BaseTest.setup(self)
    self.profile = Profile.BRAND_NEW
    self.set_profile_pref({'browser.download.dir': IrisCore.get_downloads_dir()})
    self.set_profile_pref({'browser.download.folderList': 2})
    self.set_profile_pref({'browser.download.useDownloadDir': True})
    return

The Run function

All of your test case instructions will be implemented in this function. We'll cover the basics of what you'll be doing in here a bit later.

def run(self):

The Cleanup Function

Usually cleanup isn't necessary because Iris refreshes the profile for each test. However, there may be times when your test touches things outside of a profile, eg. the downloads directory. It is here, in the teardown() that you must ensure that whatever the current test case just did won't adversely affect any test cases run after it. Any cleanup code you put here will be run if your test passes or not. For example, to cleanup the downloads folder:

def teardown(self):
    downloads_cleanup()

---

Iris Test Case Writing

Now let's take a look at the heart of a test case: the test instructions. Test case steps are essentially:

• Do something
• Verify if something happened or not
• Do another thing
• Verify if another thing happened or not

... and so forth and so on until all of the test steps have been completed.

To write Iris test cases, we ask that you take a moment to think about what it is you have to do to manually perform each test case step, and even more importantly, pay attention to how your eyes verified what happened after each action.

Let's look at a sample test to learn how to convert manual test case instructions into working Iris test step instructions. First, the manual test case:

1. Launch Firefox.
   • expected: Firefox is successfully launched.
2. Open a website that's currently not in your Bookmarks list.
   • expected: The website is properly loaded.
3. From the toolbar, click the star-shaped button.
   • expected: On click, the star-shaped button changes its color to blue.

We can ignore step 1. Launch Firefox, as that is taken care of by Iris at the start of each test case.

The code to execute steps 2 and 3 looks like:

    def run(self):
        bookmark_button_pattern = LocationBar.STAR_BUTTON_UNSTARRED

        navigate(LocalWeb.MOZILLA_TEST_SITE)

        mozilla_page_assert = exists(LocalWeb.MOZILLA_LOGO, 10)
        assert_true(self, mozilla_page_assert, 'Mozilla page loaded successfully.')

        try:
            wait(bookmark_button_pattern, 10)
            logger.debug('Bookmark star is present on the page.')
            click(bookmark_button_pattern)
        except FindError:
            raise FindError('Bookmark star is not present on the page, aborting.')

        page_bookmarked_assert = exists(Bookmarks.StarDialog.NEW_BOOKMARK, 10)
        assert_true(self, page_bookmarked_assert, 'The page was successfully bookmarked via star button.')

Let's go over each section.

The first line

        bookmark_button_pattern = LocationBar.STAR_BUTTON_UNSTARRED

is a pattern declaration of a Firefox_UI class asset. All pattern declarations should be done at the beginning of the run function.

The browser is running. Now, in step 2, we have to visit an un-bookmarked page. Iris loads a clean profile by default, so a local test page will suffice.

        navigate(LocalWeb.MOZILLA_TEST_SITE)

navigate() is a function that handles typing a URL in the awesomebar and going to the page. In this case, a local web page for a basic Mozilla-themed test site. Then, per the test case steps, we should ensure the page is correctly loaded.

        mozilla_page_assert = exists(LocalWeb.MOZILLA_LOGO, 10)
        assert_true(self, mozilla_page_assert, 'Mozilla page loaded successfully.')

The above code first checks that the Mozilla logo is loaded in the test page by using Iris' exist() function. exists() searches for the pattern (first parameter) on screen for the given number of seconds (second parameter). If the pattern is found in time, then True is returned to the variable. If the pattern is not found in time, False is assigned to the variable. The second line, the assert_true() function, checks the value of the variable. It reports "pass" plus the quoted message if true, or it reports "fail" plus the message if the assert is false.

Next, Step 3, we want to click the star-shaped button to bookmark the page. One way to do this is

        try:
            wait(bookmark_button_pattern, 10)
            logger.debug('Bookmark star is present on the page.')
            click(bookmark_button_pattern)
        except FindError:
            raise FindError('Bookmark star is not present on the page, aborting.')

The wait() function is slightly different than exists() in that wait() only returns a True value. It otherwise throws an exception which must be handled. This requires the use of the try/except block with wait(). Besides that, wait() looks for a pattern on screen for the given amount of time, just as exists() does. If wait() is true, then the message is logged via logger.debug and a mouse click() is performed on the star button pattern. If wait() doesn't find the pattern in time, the exception handler reports an error and the test is stopped.

If the click is successful, then - per the test instructions - we have to check the results: - expected: On click, the star-shaped button changes its color to blue.

So we use another pattern exists paired with an assert_true to verify the star is filled in blue.

        page_bookmarked_assert = exists(Bookmarks.StarDialog.NEW_BOOKMARK, 10)
        assert_true(self, page_bookmarked_assert, 'The page was successfully bookmarked via star button.')

The pair of functions exists() and assert_true() is the bread and butter of test step confirmation in Iris. As long as asserts report True the next steps of the test will be run. As soon as an assert reports False, Iris stops the current test, performs any teardown() if present, shuts down Firefox, then moves on to the next test.

You will likely have one or more exists/assert pairs per test case step. Checks are also often necessary to properly verify the state of the test case before taking action. Iris will only report one pass/fail to the final test run results. As such, it doesn't matter if your test case has dozens of exists/assert pairs reporting along the way. Feel free to use them liberally to make your tests solid as a rock.

---

To be continued with:

• Explanation of most common helpers
• Description of common pitfalls we have encountered