Skip to content

Latest commit

 

History

History
55 lines (39 loc) · 2.86 KB

tempest_xml_failure_examples.rst

File metadata and controls

55 lines (39 loc) · 2.86 KB
Raw

tempest xml output

On of the best features of tempest is the fact that tests are tracked with a UUID referred to in tempest as a 'idempotent id'. This feature allows us to track a test's state over time and ensure its uniqueness. In certain scenarios XML output generated by tempest can be incomplete as test entries can be omitted.

setUpClass failure example

When a setUpClass failure occurs an entry in the xml similar to the following will be observed. The class name appears in the name attribute and since it is referring to a failure in the class it does not mention a UUID for any tests. None of the tests in the class where the failure occurred will have entries in the produced xml document.

<testcase classname="" name="setUpClass (tempest.oops.this.class.is.not.in.the.TestList)" time="0.000">
    <failure type="testtools.testresult.real._StringException">Traceback (most recent call last):
       testtools.testresult.real._StringException: Traceback (most recent call last):
       File "/openstack/venvs/tempest-r14.20.0/tempest/lib/common/rest_client.py", line 829, in _error_checker
           message=message)
       tempest.lib.exceptions.ServerFault: Got server fault
       Details: An unexpected error prevented the server from fulfilling your request.
    </failure>
</testcase>

tearDownClass failure example

When a tearDownClass failure occurs an entry in the xml similar to the following will be observed. The class name appears in the name attribute and since it is referring to a failure in the class it does not mention a UUID for any tests. You can see that since the tests that are upstream of the failure dont have any child elements they are considered a pass. This behavior contradicts other testing frameworks in python.

<testcase classname="" name="tearDownClass (tempest.api.compute.admin.test_aggregates.AggregatesAdminTestJSON)" time="0.000">
    <failure type="testtools.testresult.real._StringException">Traceback (most recent call last):
        testtools.testresult.real._StringException: Traceback (most recent call last):
        File "/openstack/venvs/tempest-r14.20.0/tempest/lib/common/rest_client.py", line 829, in _error_checker
        message=message)
        tempest.lib.exceptions.ServerFault: Got server fault
        Details: An unexpected error prevented the server from fulfilling your request.

    </failure>
</testcase>
<testcase classname="tempest.api.compute.admin.test_aggregates.AggregatesAdminTestJSON" name="test_one[id-96be03c7-570d-409c-90f8-e4db3c646996]" time="0.120"/>
<testcase classname="tempest.api.compute.admin.test_aggregates.AggregatesAdminTestJSON" name="test_two[id-eeef473c-7c52-494d-9f09-2ed7fc8fc036]" time="0.373"/>