A test data generator for Robot Framework.
You know those ugly bugs that users report that somehow were missed by all your ATDD-, TDD- and exploratory testing? Those bugs that lurk in the corners of complexity of the system that you are building. Those bugs that will make users of your application hate you..
This tool is here to help you seek and destroy those bugs before the users will find them.
It gives you the ability to generate a huge number of tests that can go through a very vast number of similar (or not so similar) test scenarios.
If you know Regular Expressions: A regular expression is a pattern that represents all the possible strings that match that pattern. RoboMachine works in a similar way. A machine represents a set of tests in a compact way. This machine can be used to generate all (or part of) the tests that it represents.
If you know Model-Based Testing or automata theory: With RoboMachine you define an extended finite state machine (in a Robot Framework -like syntax) that represents a set of tests. RoboMachine also contains algorithms that can be used to generate real executable Robot Framework tests from a RoboMachine model.
If you know Combinatorial Testing or like data driven testing: With RoboMachine you can define sets of data and rules about that data, that can be used to generate data driven tests (and also keyword driven tests). This allows you to keep your data in compact sets and let the system generate the real test data vectors from there.
Here is an example machine (using combinatorial techniques) that can generate tests for SeleniumLibrary demo:
*** Settings ***
Suite Setup Open Browser ${LOGIN_PAGE_URL} googlechrome
Suite Teardown Close Browser
Test Setup Go to ${LOGIN_PAGE_URL}
Library SeleniumLibrary
*** Variables ***
${USERNAME_FIELD} username_field
${PASSWORD_FIELD} password_field
${LOGIN_BUTTON} LOGIN
${VALID_USERNAME} demo
${VALID_PASSWORD} mode
${LOGIN_PAGE_URL} http://localhost:7272/html/
*** Machine ***
${USERNAME} any of ${VALID_USERNAME} ${VALID_PASSWORD} invalid123 ${EMPTY}
${PASSWORD} any of ${VALID_PASSWORD} ${VALID_USERNAME} password123 ${EMPTY}
# Add an equivalence rule to reduce the number of generated tests
${USERNAME} == ${VALID_PASSWORD} <==> ${PASSWORD} == ${VALID_USERNAME}
Login Page
Title Should Be Login Page
[Actions]
Submit Credentials ==> Welcome Page when ${USERNAME} == ${VALID_USERNAME} and ${PASSWORD} == ${VALID_PASSWORD}
Submit Credentials ==> Error Page otherwise
Welcome Page
Title Should Be Welcome Page
Error Page
Title Should Be Error Page
*** Keywords ***
Submit Credentials
Input Text ${USERNAME_FIELD} ${USERNAME}
Input Password ${PASSWORD_FIELD} ${PASSWORD}
Click Button ${LOGIN_BUTTON}
Here is another example machine (using model-based testing with finite state machine):
*** Settings ***
Suite Setup Open Browser ${LOGIN_PAGE_URL} googlechrome
Suite Teardown Close Browser
Test Setup Go to ${LOGIN_PAGE_URL}
Library SeleniumLibrary
*** Variables ***
${USERNAME_FIELD} username_field
${PASSWORD_FIELD} password_field
${LOGIN_BUTTON} LOGIN
${VALID_USERNAME} demo
${VALID_PASSWORD} mode
${LOGIN_PAGE_URL} http://localhost:7272/html/
*** Machine ***
Login Page
Title Should Be Login Page
[Actions]
Submit Valid Credentials ==> Welcome Page
Submit Invalid Credentials ==> Error Page
Welcome Page
Title Should Be Welcome Page
[Actions]
Go to ${LOGIN_PAGE_URL} ==> Login Page
Error Page
Title Should Be Error Page
[Actions]
Go to ${LOGIN_PAGE_URL} ==> Login Page
*** Keywords ***
Submit Valid Credentials
Submit Credentials ${VALID_USERNAME} ${VALID_PASSWORD}
Submit Invalid Credentials
Submit Credentials invalid_username invalid_password
Submit Credentials
[Arguments] ${username} ${password}
Input Text ${USERNAME_FIELD} ${USERNAME}
Input Password ${PASSWORD_FIELD} ${PASSWORD}
Click Button ${LOGIN_BUTTON}
NOTE! This machine can generate infinite number of test sequences thus you need to constraint the generation. For example "robomachine --tests-max 10 --actions-max 20 --to-state 'Welcome Page' --generation-algorithm random [MACHINE_FILE_NAME]" will generate 10 random tests with at most 20 actions each and all tests ending to state 'Welcome Page'.
From Python Package Index:
pip install RoboMachine
From source:
git clone git://github.com/mkorpela/RoboMachine.git cd RoboMachine python setup.py install
After this you should have a commandline tool called 'robomachine' available. See 'robomachine --help' for commandline tool usage.
Only supported Robot Framework format is space separated format.
You should use .robomachine suffix for your Machine files.
You can have a Robot Framework Settings table at the beginning of the machine file.
You can have a Robot Framework Variables table after the optional Settings table.
Machine table is a must.
You can have a Robot Framework Keywords table after the Machine table.
Used machine variables must be introduced at the beginning of the machine table. Valid machine variable is a valid Robot Framework variable that contains only uppercase characters A-Z, numbers (not at the start) 0-9 and underscores '_'. For example: ${VALID_123} is a valid machine variable.
An example of a valid machine variable definition line:
${VALID_VARIABLE_1} any of Value 1 Value 2 Value 3
- Rules about the machine variables should be next. Rule building blocks:
- Variable condition: ${VARIABLE} == value
- Variable unequal condition: ${VARIABLE} != value
- And rule: [some condition] and [some other condition]
- Or rule: [some condition] or [some other condition]
- Not rule: not ([some condition])
- Implication rule: [some condition] ==> [some condition]
- Equivalence rule: [some condition] <==> [some condition]
- [some condition]: Variable condition OR (rule)
Rules can be used to remove variable combinations that should not be used in test generation.
An example of a valid rule line:
${VARIABLE_1} == Value 1 ==> (${FOO} == bar or ${FOO} == zoo)
Which means: When ${VARIABLE_1} value is "Value 1" then ${FOO} should either be "bar" or "zoo".
State blocks should be next. First state block is the starting state.
State block starts with a line containing the states name. Valid state name contains only upper and lowercase characters a-zA-Z, numbers (not at the start) 0-9 and spaces (not at the start or end and only one between words).
This is followed by the Robot Framework steps that should be executed when in that state.
This can be followed by an actions block definition.
An actions block starts with [Actions] tag and is followed by one or more action lines.
- An action line has four parts:
- a Robot Framework step that is executed when the action happens (action label) (you can also leave this out - use a tau transition)
- ' ==> ' right arrow with two spaces before and after
- Name of the state that the machine ends up when the action is taken (end state)
- Optional rule (when the action is available) this either starts with ' when ' and the rule or ' otherwise' - meaning this action should be taken when all of the other actions with same action label are not available
An example of a valid state definition:
State Name
Robot Framework Step with ${arguments}
Log something WARN
[Actions]
Log other thing ==> Other State when ${FOO} == bar
Log other thing ==> Another State when (${FOO} == zoo and ${BAR} == goo)
Log other thing ==> End State otherwise
Log nothing ==> End State
==> Some state # This here is a tau transition that does not write a step to the test