Allure

Allure is a test report tool that creates a nifty HTML report from test execution results. Allure also brings some Java programming features that allows you to define test steps and add parameters and attachments that can enhance the created report. See the Allure documentation for more details.

Check out their demo report

Reports are generated via Maven command mvn clean test allure:report which will clean the project, execute tests and finally generate the report from test results. Since Allure writes their own test results besides the JUnit results, the tool needs to be integrated into the test cycle. Maven Surefire is a Maven plugin which covers the unit test phase during the build lifecycle. Please get the latest supported configuration from the pom.xml of the template project and update your test project pom.xml on demand.

Structure the Allure Report using Steps

Each Allure step in the test project creates an line item within the report when executed. Such an item can be expand- and collapsable if it contain other steps. It hides all reported sub steps when it's collapsed and can be expanded to deep dive into the test execution actions. Thus the report can show grouped actions as sections of the test case execution to get a better overview about how the test case runs along the system under test. We recommend to annotate each method with the @Step annotation and add a short description make it visible in the report in case of needed investigations. You can also use the methods parameters in the description to provide all necessary information on the first look.

There are different ways to define the steps of your test case. These will appear in the report later on. The first way is to use the @Step annotation on the method that represents a step in your test:

@Step("Simple action")
public void mySimpleAction()
{
    // do something that would occur within the step "Simple action" in the report
}

@Step("Simple action with: '{parameter}'")
public void mySimpleActionWithParameter(Object parameter)
{
    // do something that would occur within the step "Simple action with: 'parameter.toString()'" in the report
    // the @Step annotation will call the toString method of the passed parameter to add the needed information to the report
}

The second way is to utilize a method that passes the step information to the annotation. Neodymium provides such methods via the AllureAddons class:

@NeodymiumTest
public void myTest1()
{
    AllureAddons.addToReport("first part");
    // do something that would occur after the step "INFO: {first part}" in the report

    AllureAddons.addToReport("second part");
    // do other things that would occur after the step "INFO: {second part}" in the report
}

If you want to wrap a complete section within a step you could use the following funtionality.

@NeodymiumTest
public void myTest2()
{
    AllureAddons.step("first part" () -> {
        // do something that would occur within the step "first part" in the report
    });

    AllureAddons.step("second part"() -> {
        // do other things that would occur within the step second part}" in the report 
    });
}

Describing your test cases with annotations

Allure also provides several other annotations that can help you enhance your test report. The following annotations need to be applied to your test case.

@Owner("Lisa Smith")
@Severity(SeverityLevel.BLOCKER)
@Tag("smoke")
@Tag("registered")
@Description("place an order as a registered shop user")
public class RegisteredOrderTest extends AbstractTest
{
}

Furthermore, you can add general, issue and test management system (TMS) links via annotation that will appear in your test report. Of course this approach would require to maintain the bugs within the test project itself which could be quiet cumbersome and quickly outdated.

@TmsLink("end-of-script-developer-and-now")
@Issue("148")
@Link(url = "https://ask.xceptance.de/t/end-of-script-developer-and-now/148", type = "custom", name = "DemoLink")
public class HomePageTest extends AbstractTest
{
}

To use the TmsLink and Issue annotations you need to specify the urls of the systems in your pom.xml as below.

<systemPropertyVariables>
    <allure.results.directory>${project.build.directory}/allure-results</allure.results.directory>
    <allure.link.issue.pattern>https://ask.xceptance.de/t/{}</allure.link.issue.pattern>
    <allure.link.tms.pattern>https://ask.xceptance.de/t/{}</allure.link.tms.pattern>
    <selenide.reports>${project.build.directory}/selenide-results</selenide.reports>
</systemPropertyVariables>

Reports

For more information on Allure reports. Look here.