StatusChecker is a tool for validating that executed Robot Framework test cases have expected statuses and log messages. It is mainly useful for Robot Framework test library developers who want to use Robot Framework to also test their libraries. StatusChecker 1.3 and newer are compatible both with Python 2 and Python 3.
The easiest way to install StatusChecker is by using pip:
pip install robotstatuschecker
python setup.py install
From the command line:
python -m robotstatuschecker infile [outfile]
from robotstatuschecker import process_output process_output('infile.xml', 'outfile.xml')
If an output file is not given, the input file is edited in place.
By default, all test cases are expected to PASS and have no
message. Changing the expected status to FAIL is done by having
FAIL (in uppercase) somewhere in the test case
documentation. The expected error message must then follow
If a test is expected to PASS with a certain message, the word
PASS must be added to its documentation explicitly and the
expected message given after that.
The expected message can also be specified as a regular expression by
prefixing it with
REGEXP:. The specified regular expression
must match the error message fully. Having spaces between the status,
the message and the possible regular expression prefix is optional.
An alternative to using regular expressions is using glob patterns where
* matches anything (including newline) and
? matches any single
character. This is can be accomplished by starting the expected message
Finally, it is possible to test that the message starts with something
by prefixing the expected message with
The following examples illustrate different ways to define test statuses and messages:
*** Test Cases *** Simple failure [Documentation] FAIL Expected error message Steps Exclude documentation before marker [Documentation] This text is ignored FAIL Expected error message Steps Regexp example [Documentation] FAIL REGEXP: (IOError|OSError): .* Steps Glob example [Documentation] FAIL GLOB: ??Error: * Steps Start example [Documentation] FAIL STARTS: IOError: Steps Passing without message Steps Passing with message [Documentation] PASS Expected message Steps
The expected keyword log messages can also be defined in the test case documentation using a syntax such as:
LOG x.y:z LEVEL Actual message
The part before the colon specifies the keyword to check. For
1 means first keyword,
1.2 is the second child
keyword of the first keyword, and so on.
The part after the colon species the message. For example,
means the second message of the first keyword and
the third message of the second child keyword of the first keyword.
The message index is optional and defaults to
Message level is specified before the actual message, and it can be
any of the valid log levels in capital letters. If the level is not
given it defaults to
Possible leading and trailing whitespace is ignored both in the expected and in the actual log message.
This syntax can be used multiple times to test multiple messages. It
also works together with specifying the expected error message with
FAIL, but it that case
FAIL and the expected error must
It is also possible to give the message as a regular expression or glob
pattern or to give just the start of the message. This is accomplished
by prefixing the message with
respectively, exactly like when defining expected test status.
Finally, to check that a keyword does not have a certain message, it
is possible to use
NONE in the place of the message.
*** Test cases *** Simple example [Documentation] LOG 1 Hello, world! Steps Nested keywords [Documentation] LOG 2.1 1st child of 2nd kw Steps Message index [Documentation] LOG 2:2 2nd msg of 2nd kw Steps Nested and index [Documentation] LOG 3.1:2 2nd msg of 3rd kw's 1st child Steps Log levels [Documentation] LOG 2 DEBUG Debug-level message ... LOG 1.2:3 WARN Warning Steps Multiple messages [Documentation] LOG 1 First tested message ... LOG 1.2 Second tested message ... LOG 2.2.1 DEBUG Third tested message Steps Status and log [Documentation] FAIL Expected error message ... LOG 1.2 Expected log message Steps Regexp message [Documentation] LOG 1 REGEXP: (Hello|Hi) world! Steps Glob message [Documentation] LOG 1 GLOB: * world! Steps Start of the message [Documentation] LOG 1 STARTS: Hello w Steps No message [Documentation] LOG 1:1 Test that we have only 1 msg ... LOG 1:2 NONE Steps