Tutorial 8: Model based testing
Model-based testing is a grey-box testing approach that provides coverage based on a model (a limited-knowledge representation) of the system-under-test. In this course (and for the purpose of this tutorial), we will use a state model (statechart) of the system-under-test for model-based testing purposes.
GraphWalker is an open-source model-based testing tool. Users would use GraphWalker:
- to create a statechart that they would like to test (in our case, using the GraphWalker Studio editor).
- to generate paths through the statechart according to specified algorithms and end conditions.
- to verify the correctness of the statechart implementation.
GraphWalker also offers a Maven plug-in for integration into Java Projects.
In this tutorial, we are given a system specification written in natural language. We will:
- Create a corresponding statechart using GraphWalker.
- Integrate the statechart into a Maven project. This will involve mapping the statechart components to Java.
- Run tests on the Maven project using GraphWalker.
This section contains instructions on how to use GraphWalker Studio to create a statechart for the system-under-test (SUT) described below.
The system-under-test (SUT) is a car door. It is initially in a closed state. A user may open the door, which brings it to an opened state. A user may close an open door, which brings it back to a closed state. A user may lock a closed door to make it a locked door. A locked door may be unlocked by the user, which would bring it back to a closed state.
- Download the
graphwalker-studio-4.3.1.jarfile located on the GraphWalker GitHub page. - Run the
.jarfile using the following command:
java -jar graphwalker-studio-4.3.1.jar
- Navigate to http://localhost:9090/studio.html to access the GraphWalker Studio.
-
Creating an empty model: Click on the
button at the top left to create an empty model. This opens the editor view. - Creating a vertex: In the editor view area, while holding the keyboard key v, click the left mouse button.
-
Creating an edge: In the editor view area, while holding the keyboard key e,
- click and hold the left mouse button on the source vertex.
- drag the mouse cursor to the target vertex and release the left mouse button
-
Adjusting the element names: In the editor view area,
- select the element you wish to rename.
- open the side bar by clicking on the
button. - rename the element.
-
Selecting the start element: In the editor view area,
- select the start element.
- open the side bar by clicking on the button.
- toggle the Start element switch.
NOTE: You may rename your model to something more relevant, like CarDoorModel.
SAMPLE SOLUTION: A picture of a sample solution for the System Specification described above is located here.
To do so, click on the 
play button located at the left side of your window. This will start an animation that shows that GraphWalker is traversing your statechart.
Once your statechart is completed, you should export a corresonding .json file by clicking on the 
save icon at the left of your window.
To start your implementation, you should clone the template graphWalker project located here. This repository contains two directories. The com.company.carDoor/ folder contains the java implementation of the system specified in the previous section. The com.company.graphwalker/ folder contains a template repository where we will integrate the GraphWalker implementations.
After cloning the repository, run the following commands to ensure that all the dependencies are installed properly:
cd carDoor-graphwalker-template
mvn install
NOTE: Optionally, you may start a new GraphWalker Maven project from scratch (instead of using the template provided in the com.company.graphwalker/ directory) by running the command below.
mvn archetype:generate -B \
-DarchetypeGroupId=org.graphwalker \
-DarchetypeArtifactId=graphwalker-maven-archetype \
-DgroupId=com.company -DartifactId=myProject \
-DarchetypeVersion=LATEST
As a first step, you should add the .json file you generated by GitHub Studio into the com.company.graphwalker/src/main/resources/com/company/ directory. You should delete the exisiting insertJsonStatechartHere.txt file as it is just a placeholder.
We are now ready to start working with GraphWalker. While in the repository source folder, run the following commands to generate Java source code from your .json file:
cd com.company.graphwalker
mvn graphwalker:generate-sources
This will generate a Java interface in the target/generated-sources/graphwalker/com/company/ folder. The Java interface should look like the code snippet below (it should contain all the vertices and edges you included in your GraphWalker Studio Statechart).
// Generated by GraphWalker (http://www.graphwalker.org)
package com.company;
import org.graphwalker.java.annotation.Model;
import org.graphwalker.java.annotation.Vertex;
import org.graphwalker.java.annotation.Edge;
@Model(file = "com/company/CarDoorModel.json")
public interface CarDoorModel {
@Vertex()
void v_Closed();
@Edge()
void e_unlock();
@Vertex()
void v_Opened();
@Edge()
void e_lock();
@Vertex()
void v_Locked();
@Edge()
void e_close();
@Edge()
void e_open();
}
Now that you have generated the Java interface, you must create an implementing class to be able to run GraphWalker tests.
This implementing class will act as a mapping between GraphWalker and the SUT implementation provided in the com.company.carDoor directory.
A partially written implementing class is provided in the com.company.graphwalker/src/main/java/com/company/CarDoortest.java file. You must complete the file by adding the missing implementations for the abstract methods included in the generated Java interface. You may also need to modify some of the class and method names provided in the partial file such that they correspond to the naming you have used in your GraphWalker Studio model.
When writing the implementing class, it is important to keep in mind the semantics of edges and vertices:
- Edges represent operations performed on the system. Therefore, an edge definition should contain a function call to the corresponding SUT function.
- Vertices represent states of the statechart. When the traversal reaches a vertex, we should make assertions to check whether the SUT is behaving as explained in the specification.
With that having been said, the final implementation of the generated Java interface should look like the code snippet bellow. Note that each edge method (with a name that starts with "e_") contains a single function call, while each vertex method (with a name that starts with "v_") contains an assertion that checks the current state. Important: A vertex method may also contain other assertions that pertain to other information about the SUT, if such information is provided in the System Specification.
package com.company;
import org.graphwalker.core.machine.ExecutionContext;
import org.graphwalker.java.annotation.GraphWalker;
import org.junit.Assert;
@GraphWalker(value = "random(edge_coverage(100))", start = "v_Closed" )
public class CarDoorTest extends ExecutionContext implements CarDoorModel {
private final CarDoor door = new CarDoor();
@Override
public void e_unlock(){
System.out.println("e_unlock");
door.unlockDoor();
}
@Override
public void e_lock(){
System.out.println("e_unlock");
door.lockDoor();
}
@Override
public void e_close(){
System.out.println("e_unlock");
door.closeDoor();
}
@Override
public void e_open(){
System.out.println("e_unlock");
door.openDoor();
}
@Override
public void v_Closed(){
Assert.assertTrue(door.getStatus() == CarDoorStatus.CLOSED);
System.out.println("v_Closed");
}
@Override
public void v_Opened(){
Assert.assertTrue(door.getStatus() == CarDoorStatus.OPENED);
System.out.println("v_Opened");
}
@Override
public void v_Locked(){
Assert.assertTrue(door.getStatus() == CarDoorStatus.LOCKED);
System.out.println("v_Locked");
}
}
After creating the implementing class for the generated Java interface, the simplest way to run GraphWalker tests is to do so directly through Maven. This can be done by running the following command while in the com.company.carDoor directory:
mvn graphwalker:test
This will ask GraphWalker to traverse your statechart according to a generator strategy until the stop condition is reached. These parameters, as well as the start vertex, are defined in the @GraphWalker(value = "random(edge_coverage(100))", start = "v_Closed" ) annotation included in the class you implemented earlier.
For this specific example, GraphWalker will use random edge selection as a generator strategy for statechart traversal, and will end the traversal once it reaches 100% edge coverage. An extensive list of generators and stop conditions is provided in the GraphWalker Documentation.
Running the above command will output two things:
- A trace of the statechart traversal, in the following format:
e_unlock
02:58:43.679 [main] DEBUG org.graphwalker.core.machine.SimpleMachine - Context: com.company.CarDoorTest@725a790e
02:58:43.679 [main] DEBUG org.graphwalker.core.machine.ExecutionContext - Execute method: 'v_Closed' in model: 'CarDoorModel'
v_Closed
02:58:43.681 [main] DEBUG org.graphwalker.core.machine.SimpleMachine - Context: com.company.CarDoorTest@725a790e
02:58:43.681 [main] DEBUG org.graphwalker.core.machine.ExecutionContext - Execute method: 'e_open' in model: 'CarDoorModel'
e_unlock
02:58:43.682 [main] DEBUG org.graphwalker.core.machine.SimpleMachine - Context: com.company.CarDoorTest@725a790e
02:58:43.683 [main] DEBUG org.graphwalker.core.machine.ExecutionContext - Execute method: 'v_Opened' in model: 'CarDoorModel'
v_Opened
02:58:43.684 [main] DEBUG org.graphwalker.core.machine.SimpleMachine - Context: com.company.CarDoorTest@725a790e
02:58:43.684 [main] DEBUG org.graphwalker.core.machine.ExecutionContext - Execute method: 'e_close' in model: 'CarDoorModel'
e_unlock
02:58:43.686 [main] DEBUG org.graphwalker.core.machine.SimpleMachine - Context: com.company.CarDoorTest@725a790e
02:58:43.686 [main] DEBUG org.graphwalker.core.machine.ExecutionContext - Execute method: 'v_Closed' in model: 'CarDoorModel'
- Some statistics of the graphWalker run:
Result :
{
"totalFailedNumberOfModels": 0,
"totalNotExecutedNumberOfModels": 0,
"totalNumberOfUnvisitedVertices": 0,
"verticesNotVisited": [],
...
}
Another way to run GraphWalker is through a main method. Such a method is already provided in the com.company.graphwalker/src/main/java/com/company/Runner.java file. To run GraphWalker though a main method, you can run the following command while in the com.company.carDoor directory:
mvn compile exec:java -Dexec.mainClass="com.company.Runner"
From the perspective of GraphWalker, both approaches indicated above do the same thing. However, the programatic approach allows for more customizeability and control in what exactly you want to run and what kind of data you want to output. For example, if you want to run Graphwalker on multiple different statecharts, you can simply add the corresponding classes to the TestExecutor object located in the Runner.java file.
A fully implemented sample solution for this tutorial is located here.