-
Notifications
You must be signed in to change notification settings - Fork 2
Comparison tests
WojciechRynczuk edited this page Feb 5, 2017
·
1 revision
Comparison tests are the application level functional tests. Running the test consists of two steps:
- Invoking the tested application with the predefined parameters and source files
- Comparing the output file with the test reference file. If the output file and the reference file are matching the test passes.
The test parameters are provided by the test.xml file which is application specific and it is located in the 'test' subfolder along with input and reference files. The command-line parameters passed during the application execution are generated upon the tags values in this file.
The expected XML structures have been described below.
<test>
<maker>
<info>
<name>Test name</name>
<description>Brief test description</description>
</info>
<common>
<verbose>False</verbose>
<output_file>output_filename</output_file>
<golden_file>golden_filename</golden_file>
</common>
<unique>
<input_file>input_filename</input_file>
<time_unit>us</time_unit>
<line_counter>counter</line_counter>
</unique>
</maker>
</test>
<test>
<merge>
<info>
<name>Test name</name>
<description>Brief test description</description>
</info>
<common>
<verbose>False</verbose>
<output_file>output_filename</output_file>
<golden_file>golden_filename</golden_file>
</common>
<unique>
<time_unit>output_time_unit</time_unit>
<sources>
<source>
<format>T</format>
<time_stamp>71050601</time_stamp>
<time_unit>us</time_unit>
<prefix>System1</prefix>
<line_counter>Counter</line_counter>
<input_file>input1_filename</input_file>
</source>
<source>
<format>T</format>
<time_stamp>234256037</time_stamp>
<time_unit>us</time_unit>
<prefix>System2</prefix>
<line_counter>Counter</line_counter>
<input_file>input2_filename</input_file>
</source>
</sources>
</unique>
</merge>
</test>
The functional tests can be invoked separately for each of the applications.
Syntax: comparison_test.py -e executable_path -t test_directory_path [-v]
Parameters:
-
-e
or--exec
the path to the tested executable, -
-t
or--testdir
the path to the test directory containing the test.xml file, -
-v
or--verbose
the optional verbose mode provides additional test information.