How to write a new test case
This document explains how to write a new test case. As you may already have noticed, the tests are actually simple Python Unittests. As from Fedora 23 our Cloud Base and Atomic images have only Python 3, we write our testcases in Python 3 only.
How they work?
While we do manual testing of some project, we generally run some command, and check the output, or exit code to verify if the command did what it was supposed to do. We do the same but using Python unittests. If you see any of the testcases' Python source code, you will find that we import two functions
from .testutils import system, if_atomic
The system function can execute a terminal command, it returns the output, any error message, and the exit code of the command given. We use if_atomic to function to skip or run tests particularly for an Atomic image.
Anatomy of a test case
Say we have to check if docker rpm is installed in the Atomic image. We can do it manually using rpm command in the following way.
rpm -q docker
If the docker package is not installed, it will say that in the output of the command.
package docker is not installed
So let us convert this into a Python test case.
@unittest.skipUnless(if_atomic(), "It's not an Atomic image") class TestDockerInstalled(unittest.TestCase): def test_run(self): out, err, eid = system('rpm -q docker') out = out.decode('utf-8') self.assertFalse('not installed' in out, out)`
As you can see, we are converting back the out to a string from bytes using decode method. There are many such test example in the atomictests or nongatingtests file. If you need any help, feel free to ping trishnag, fhackdroid or kushal on #fedora-cloud channel.
How to execute the tests?
Remember to use Tunir version 0.12 or later, you can get them in Fedora updates-testing repo.
In the git repo we have a localrunner.sh which creates a tarball of the tests, and runs a webserver at port 8000 using Python SimpleHTTPServer module. So run it from one terminal. You can stop the server by pressing Ctrl + C. In tunir a job is defined inside jobname.txt, you can see an example in fedora.txt file.
curl -O http://192.168.122.1:8000/tunirtests.tar.gz tar -xzvf tunirtests.tar.gz sudo python3 -m unittest tunirtests.atomictests.TestDockerInstalled -v ## sudo python3 -m unittest tunirtests.nongatingtests.TunirNonGatingtestsCpio -v ## sudo python3 -m unittest tunirtests.nongatingtests.TunirNonGatingtests -v
In the first line, we download the tarball.
The second command untars the tarball.
From third line we execute the actual tests, if you want to reboot the system, you can do it by
sudo reboot command. After any reboot, remeber to add a
SLEEP 60 like command to sleep for 60 seconds (or any time based on your system's performance).
If a command returns non zero exit code, you can mark it with
@@ in the beginning of the command. If it fails to return a non-zero exit code, tunir will mark that command as failed, and stop the job there. We can also have non-gating tests, if any command starts with
## , tunir marks it as non-gating. Non-gating means even if the test fail tunir will continue the commands/tests defined in the jobname.txt. At the end of tunir job it prints the details about non-gating tests like.
Non gating tests status: Total:2 Passed:2 Failed:0
The home page contains the currently activated tests in autocloud.