Skip to content

Latest commit

 

History

History
84 lines (60 loc) · 4.87 KB

test_dependency.rst

File metadata and controls

84 lines (60 loc) · 4.87 KB
Raw

Test Dependency

Note

This feature is subject to change

Buildtest can support test dependencies which allows one to specify condition before running a test. Let's take a look at this next example, we have a buildspec with three tests jobA, jobB, and jobC. The test jobA will run immediately but now we introduce a new keyword needs which is a list of test names as dependency. We want test jobB to run after jobA is complete, and jobC to run once jobA and jobB is complete.

.. literalinclude:: ../tutorials/job_dependency/ex1.yml
   :language: yaml
   :emphasize-lines: 2,10,14,19,23

The needs property expects a list of strings, and values must match name of test. If you specify an invalid test name in needs property then buildtest will ignore the value. If multiple tests are specified in needs property then all test must finish prior to running test.

Let's run this test, and take a note that buildtest will run test jobA, followed by jobB then jobC.

.. dropdown:: ``buildtest build -b tutorials/job_dependency/ex1.yml``

   .. command-output:: buildtest build -b tutorials/job_dependency/ex1.yml

Test Dependency by returncode

In this next example, we can control behavior of job dependency based on returncode for a given test. This test has three tests: test1, test2 and test3. The first test will exit with returncode 1 but this test will pass because we have set state: PASS to override the status check. The next test test2 requires test1 to have a returncode of 1 in order to satisfy dependency. The returncode property expects a valid returncode and it can be a list of returncode similar to how one specify returncode under the status property see :ref:`returncode`. The needs property can support multiple test with returncode, in test3 we require test1 to have returncode 1 while test2 has a returncode of 2. We expect test2 to return a returncode of 2 because of exit 2 statement so we expect all three tests to run.

.. literalinclude:: ../tutorials/job_dependency/ex2.yml
   :language: yaml
   :emphasize-lines: 2,7-8,10,17-19,21,28-32

Let's build this test and take note of execution order of test.

.. dropdown:: ``buildtest build -b tutorials/job_dependency/ex2.yml``

   .. command-output:: buildtest build -b tutorials/job_dependency/ex2.yml

Test Dependency by state

You can specify state as a property to check for test state when specify test dependency. In this next example, we have have four tests pass_test, fail_test, pass_and_fail_test, and final_test. The first test will be a PASS because we have state: PASS. The test fail_test depends on pass_test only if it has state: PASS, if value is mismatch then test will be skipped. Note that buildtest will skip test until next iteration if test is not executed, however if test is complete then buildtest will cancel dependent test. We can specify multiple test dependencies with state property such as test pass_and_fail_test which expects pass_test to have state: PASS and fail_test to have state: FAIL. In test final_test, shows how you can combine the format, the needs property is a list of object where each element is name of test. If no properties are associated with test name then buildtest will wait until job is complete to execute test. In this example, the test expects both pass_test and fail_test to run while pass_and_fail_test must have returncode of 1.

.. literalinclude:: ../tutorials/job_dependency/ex3.yml
   :language: yaml
   :emphasize-lines: 2,6-7,12,18-20,25,29-33,40,44-48

Let's build this test and take note all tests are run.

.. dropdown:: ``buildtest build -b tutorials/job_dependency/ex3.yml``

   .. command-output:: buildtest build -b tutorials/job_dependency/ex3.yml

In this next example, we have three tests the first test will runtime_test will sleep for 5 seconds but it will fail due to runtime requirement of 2sec. The next two tests runtime_test_pass and runtime_test_fail both depend on runtime_test however due to condition only one of them can be run because runtime_test_pass expects runtime_test to have state: PASS while runtime_test_fail expects runtime_test to have state: FAIL. This type of workflow can be used if you want to run a set of test based on one condition while running a different set of test based on the negative condition.

.. literalinclude:: ../tutorials/job_dependency/ex4.yml
   :language: yaml
   :emphasize-lines: 2,6-8,11,15-17,20,24-26

Let's build this test and take note that we only run two tests and runtime_test_fail was skipped because test runtime_test has a state: PASS.

.. dropdown:: ``buildtest build -b tutorials/job_dependency/ex4.yml``

   .. command-output:: buildtest build -b tutorials/job_dependency/ex4.yml