Serenity report does not contain scenario html pages

[aybartsch]

What happened?

I'm running serenity cucumber tests from within main class inside of a fat jar programatically:

    private static TestExecutionSummary getTestExecutionSummary(ClassSelector clazz) {
        final LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request().selectors(clazz).build();
        final Launcher launcher = LauncherFactory.create();
        final SummaryGeneratingListener listener = new SummaryGeneratingListener();
        launcher.registerTestExecutionListeners(listener);
        launcher.execute(request);
        return listener.getSummary();
    }

Afterwards the serenity report is created as below:

    private static void generateTestReports() {
        HtmlAggregateStoryReporter reporter = new HtmlAggregateStoryReporter(Serenity.getDefaultProjectKey());
        File sourceDirectory = new File("target/site/serenity/");
        reporter.setOutputDirectory(sourceDirectory);
        try {
            reporter.generateReportsForTestResultsFrom(sourceDirectory);
        } catch (IOException ioException) {
            // Some exception logging
        }
    }

Test classes are annotated this way:

@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("/features/basic/LoginCentral.feature")
@ConfigurationParameter(key = PLUGIN_PROPERTY_NAME,
        value = "io.cucumber.core.plugin.SerenityReporterParallel,pretty")
@ConfigurationParameter(key = GLUE_PROPERTY_NAME,
        value = "com.westernacher.ztr.e2e.steps.basic")
public class LoginCentralTest extends BaseTest {
}

To be able to run the above test case from within fat jar it was necessary to apply the below dependencies:

    testImplementation "net.serenity-bdd:serenity-core:$serenityVersion"
    testImplementation "net.serenity-bdd:serenity-junit5:$serenityVersion"
    testImplementation "net.serenity-bdd:serenity-cucumber:$serenityVersion"
    testImplementation "net.serenity-bdd:serenity-rest-assured:$serenityVersion"
    testImplementation "io.github.bonigarcia:webdrivermanager:$webdriverManagerVersion"
    testImplementation "org.junit.jupiter:junit-jupiter:$junitVersion"
    testImplementation "org.junit.platform:junit-platform-launcher:$junitPlatformVersion"
    testImplementation "org.junit.platform:junit-platform-suite-engine:$junitPlatformVersion"
    testImplementation "org.junit.platform:junit-platform-console-standalone:$junitPlatformVersion"
    testImplementation "io.cucumber:cucumber-junit-platform-engine:$cucumberVersion"

Unfortunately, the serenity report is not complete i.e. I can open the overview page but no scenario pages. They are simply missing the "target/site/serenity" folder. When running the main class in IDE the report is complete. Thus, I have some doubts about the usage of junit-platform-console-standalone.

What did you expect to happen?

The report should be complete.

Serenity BDD version

4.0.15

JDK version

11.0.21

Execution environment

macOS 12.7.1
Chrome 118

How to reproduce the bug.

• Create a fat jar from test project
• Run the fat jar
• Check the report

How can we make it happen?

Add it to the Serenity BDD backlog and wait for a volunteer to pick it up

[wakaleo]

Running the tests from a fat Jar is not a feature Serenity supports currently.

[aybartsch]

@wakaleo That's the answer I was afraid of ;) Do you have any idea why some html files weren't be created?

(When using earlier versions of serenity and junit 4 runners the report was still complete.)

[wakaleo]

I don't really know, probably something to do with class loaders and classpaths - I would need to study the project itself to have an idea.

[aybartsch]

@wakaleo The problem is caused by calling HtmlAggregateStoryReporter programatically. When I'm using the serenity-gradle-plugin's aggregate task the report is complete. I've seen that the HtmlAggregateStoryReporter can be called with a number of different parameters. I'm just using the project key currently. Could there be other parameters which are needed?

[aybartsch]

@wakaleo Great news! I found a simple solution for the reporter. I had a look to the AggregateTask and found one method call which I have added to the reporter:

reporter.setGenerateTestOutcomeReports();

Now, the reports are complete! :-)