Serenity test reports not getting generated properly after updating to v3.2.5

[sandeepraina-lenze]

Hello,

I have used the sample Serenity-Cucumber-Starter project and updated the Serenity Core version to v3.2.5 from v3.2.0.

Since then the reports are not getting generated properly.

Providing the project link for reference https://github.com/sandeepraina-lenze/serenity-cucumber-starter.git

Attaching the generated report for reference

target.zip

Error message observed in the logs

Failed to load requirements: src\test\resources\features
java.io.UncheckedIOException: src\test\resources\features
at org.apache.commons.io.FileUtils.listFilesAndDirs(FileUtils.java:2184)
at net.thucydides.core.files.TheDirectoryStructure.maxDepth(TheDirectoryStructure.java:56)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.getRequirementTypeOf(FileSystemRequirementsTagProvider.java:655)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.requirementFromDirectoryName(FileSystemRequirementsTagProvider.java:643)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.readRequirementFrom(FileSystemRequirementsTagProvider.java:554)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.capabilitiesAndStoriesIn(FileSystemRequirementsTagProvider.java:187)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.getRequirements(FileSystemRequirementsTagProvider.java:173)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.lastRequirementFrom(FileSystemRequirementsTagProvider.java:455)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.parentRequirementFromPackagePath(FileSystemRequirementsTagProvider.java:348)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.getParentRequirementOf(FileSystemRequirementsTagProvider.java:333)
at net.thucydides.core.requirements.ParentRequirementsProvided.forOutcome(ParentRequirementsProvided.java:26)
at net.thucydides.core.requirements.BaseRequirementsService.getParentRequirementOf(BaseRequirementsService.java:139)
at net.thucydides.core.requirements.BaseRequirementsService.getParentRequirementFor(BaseRequirementsService.java:49)
at net.thucydides.core.requirements.reports.ScenarioOutcomes.outcomeFrom(ScenarioOutcomes.java:75)
at net.thucydides.core.requirements.reports.ScenarioOutcomes.outcomeFrom(ScenarioOutcomes.java:57)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at net.thucydides.core.requirements.reports.ScenarioOutcomes.from(ScenarioOutcomes.java:28)
at net.thucydides.core.reports.html.RequirementsOverviewReportingTask.generateReports(RequirementsOverviewReportingTask.java:159)
at net.thucydides.core.reports.html.ReportExecutor.call(ReportExecutor.java:33)
at net.thucydides.core.reports.html.ReportExecutor.call(ReportExecutor.java:12)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.nio.file.NoSuchFileException: src\test\resources\features
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
at java.base/sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:199)
at java.base/java.nio.file.Files.readAttributes(Files.java:1851)
at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:220)
at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:277)
at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:323)
at java.base/java.nio.file.Files.walkFileTree(Files.java:2804)
at org.apache.commons.io.FileUtils.listAccumulate(FileUtils.java:2076)
at org.apache.commons.io.FileUtils.listFilesAndDirs(FileUtils.java:2179)
... 44 more

Please let me know if any additional detail required

[wakaleo]

I am unable to reproduce this, but it is complaining that the src/test/resources/features is no longer available, so maybe an OS or environment-specific issue?

[sandeepraina-lenze]

Thanks @wakaleo for looking into this. I made few changes in the structure and ran the tests through maven instead of gradle.
This resolved my problem and reports are getting generated at the moment.

[imjana]

The issue still exists if you run with Gradle. A workaround is to add the below config in build.gradle
serenity {
testRoot = "net.serenitybdd.demos.todos.features"
requirementsBaseDir = "src/test/resources/features"
}

tested and works as expected

[wakaleo]

Thanks for posting the workaround. Maybe have a look at the Gradle plugin code to see if you can spot why the property is not being picked up as a default value?

[jaimemonsalvo]

Thanks @wakaleo for looking into this. I made few changes in the structure and ran the tests through maven instead of gradle. This resolved my problem and reports are getting generated at the moment.

all bugs were fixed, when I switched from gradle to maven

[vx-bill]

@wakaleo This issue still exists when building with Gradle. When using the workaround posted by @imjana, there is no longer an error in the logs, however the report generation issue reported by sandeepraina-lenze still persists. Can we reopen this issue?

Steps to reproduce

1. clone serenity-cucumber-starter
2. update plugin to 3.3.4
3. run ./gradlew test
4. observe stack trace in output
5. observe reports generated differently from maven (broken links, missing tabs - as reported above)

using net.serenity-bdd:serenity-gradle-plugin:3.2.0 - there is no stack trace, however the reports still contain broken links.

Note: Tested on a windows box, will test later on a different OS.

Initial Investigation

AggregateTask.groovy (45) - Configures a FileSystemRequirementsTagProvider with this.topLevelDirectory = "src\test\resources\features".

AggregateTask.groovy (58) - Eventually calls: FileUtils.listFilesAndDirs(rootDirectory, new NotFileFilter(TrueFileFilter.INSTANCE), normalDirectoriesOnly());

rootDirectory is set to: "src\test\resources\features" (retrieved from the initially configured FileSystemRequirementsTagProvider)
As rootDirectory is at this stage a relative directory, FileUtils will throw the exception reported.

Any pointers on where to start looking for the problem? Obviously passing the relative directory is not desired, however where should this be converted from relative to a absolute path, or is it even desirable that this path of the code be executed?

[wakaleo]

This seems to be a Windows/Gradle-specific issue, as I can't reproduce it on Mac or Linux, or with Maven. In the past Gradle has caused many problems related to finding resource paths (the base directory isn't the current project directory, which causes a lot of problems), so I suspect this is related to that (the directories resolve correctly with Maven). I won't have any time to investigage this further any time soon, though - I don't work on Windows so can't really troubleshoot. However the best would be to clone the gradle plugin project and debug it locally, and propose a PR when you find a fix.

[vx-bill]

@wakaleo Thanks for the info. I agree, it does seem to be a windows/gradle issue.
I will dig a little further when I have time, however I am not sure what the correct target state should be. Even without errors, the other OS/build combinations are producing slightly different reports for the same project

[wakaleo]

@wakaleo Thanks for the info. I agree, it does seem to be a windows/gradle issue. I will dig a little further when I have time, however I am not sure what the correct target state should be. Even without errors, the other OS/build combinations are producing slightly different reports for the same project

What variations are you seeing on other OS/build combinations?

[chanduguthikonda]

The issue still exists if you run with Gradle. A workaround is to add the below config in build.gradle serenity { testRoot = "net.serenitybdd.demos.todos.features" requirementsBaseDir = "src/test/resources/features" }

tested and works as expected

@imjana I'm facing the same issue on Windows machine and my Serenity version is 3.2.5. I'm getting the same "Failed to load requirements: src\test\resources\features" repeatedly at the end of the test execution for almost an hour before the test execution completes.
I tried adding testRoot and requirementsBaseDir properties to build.gradle file but I observed that these properties are not even identified in build.gradle (I mean not getting any reference links to both the properties).
Not sure if I'm missing anything here.
Appreciate any pointers.

[wakaleo]

Fixed in 3.3.10

[chanduguthikonda]

3.3.10

@wakaleo Thanks for the update. I'm not seeing "Failed to load requirements: src\test\resources\features" error anymore after upgrading to 3.3.10.
After upgrading, I was not able to use 'net.serenity-bdd.aggregator' plugin and replaced with 'net.serenity-bdd.serenity-gradle-plugin' as part of apply plugin in my build.gradle file.
After running the tests, index.htm is not getting generated.
Am I missing anything configuration here?
Here is my build.gradle:
`defaultTasks 'clean', 'test', 'aggregate'

repositories {
mavenLocal()
jcenter()
mavenCentral()
}
buildscript {
repositories {
mavenLocal()
jcenter()
maven {
url "https://plugins.gradle.org/m2/"
}
}
dependencies {
classpath ("net.serenity-bdd:serenity-gradle-plugin:3.3.10")
}
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'net.serenity-bdd.serenity-gradle-plugin'

sourceCompatibility = 1.8
targetCompatibility = 1.8

ext {
serenityVersion = '3.3.10'
junitVersion = '4.13.1'
assertJVersion = '3.23.1'
logbackVersion = '1.2.10'
}

dependencies {
testCompile "net.serenity-bdd:serenity-core:${serenityVersion}",
"net.serenity-bdd:serenity-cucumber:${serenityVersion}",
"net.serenity-bdd:serenity-screenplay:${serenityVersion}",
"net.serenity-bdd:serenity-screenplay-webdriver:${serenityVersion}",
"net.serenity-bdd:serenity-rest-assured:${serenityVersion}",
"net.serenity-bdd:serenity-screenplay-rest:${serenityVersion}",
"junit:junit:${junitVersion}",
"org.assertj:assertj-core:${assertJVersion}",
"ch.qos.logback:logback-classic:${logbackVersion}"
}

test {
testLogging.showStandardStreams = true
systemProperties System.getProperties()
maxParallelForks 5
}
serenity {
testRoot = "java"
requirementsDir = "src/test/resources/features"

}
gradle.startParameter.continueOnFailure = true
test.finalizedBy(aggregate)

`

[sandeepraina-lenze]

Upgrade to version 3.4.1 this should resolve your issue

[chanduguthikonda]

@sandeepraina-lenze I've upgraded to 3.4.1 but still seeing the same behavior. Tests are executing fine but index.html is missing.
If I use apply plugin: 'net.serenity-bdd.aggregator' from 3..2.5, index.html is getting generated with the aggregate Serenity BDD report. But after upgrading to later versions including 3.3.10 and 3.4.1 I'm using apply plugin: 'net.serenity-bdd.serenity-gradle-plugin' as bdd aggregator is no more available in the latest versions and missing index.html.
May I have your build.gradle file. Just want to make sure I'm using the right versions.
Thank you

[wakaleo]

All the versions should be 3.4.1. Check the starter projects to compare.

[chanduguthikonda]

@wakaleo I have all the versions set to 3.4.1 as mentioned in the starter projects. Below is my build.gralde
`defaultTasks 'clean', 'test', 'aggregate'

repositories {
mavenLocal()
jcenter()
mavenCentral()
}
buildscript {
repositories {
mavenLocal()
jcenter()
maven {
url "https://plugins.gradle.org/m2/"
}
}
dependencies {
classpath ("net.serenity-bdd:serenity-gradle-plugin:3.4.1")
}
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'net.serenity-bdd.serenity-gradle-plugin'

sourceCompatibility = 1.8
targetCompatibility = 1.8

ext {
serenityVersion = '3.4.1'
junitVersion = '4.13.1'
assertJVersion = '3.23.1'
logbackVersion = '1.2.10'
}

dependencies {
testCompile "net.serenity-bdd:serenity-core:${serenityVersion}",
"net.serenity-bdd:serenity-cucumber:${serenityVersion}",
"net.serenity-bdd:serenity-screenplay:${serenityVersion}",
"net.serenity-bdd:serenity-screenplay-webdriver:${serenityVersion}",
"net.serenity-bdd:serenity-rest-assured:${serenityVersion}",
"net.serenity-bdd:serenity-screenplay-rest:${serenityVersion}",
"junit:junit:${junitVersion}",
"org.assertj:assertj-core:${assertJVersion}",
"ch.qos.logback:logback-classic:${logbackVersion}"
}

test {
testLogging.showStandardStreams = true
systemProperties System.getProperties()
maxParallelForks 5
}
serenity {
testRoot = "java"
// Specify the root directory of any Cucumber feature files
requirementsDir = "src/test/resources/features"

}
gradle.startParameter.continueOnFailure = true
test.finalizedBy(aggregate)

`

[Moncada25]

Hello @wakaleo, I have this same error, the Serenity report is generated, but with some broken links, I see that it is not indexed correctly, I am using:

Mac OS 13.2.1 (I also tested on Windows 11)
Serenity 3.6.7
net.serenity-bdd.serenity-gradle-plugin 3.6.7
Java 11 (I also did tests with Kotlin using build.gradle.kts)

I tested different versions of Serenity (using the same version of the gradle plugin) and the last one that worked correctly for me was 3.2.0

I am using these routes.

serenity {
testRoot="fulfillment-automation-common" //project name
requirementsBaseDir = "src/test/resources/features"
}

Can we reopen this case please?

[wakaleo]

Happy to reopen. I have no timeline to look at this though - if you need a fix quickly the best would be to dig into the Gradle plugin code and see what you can do. There are known issues with the way Gradle calculates the working directory which causes problems with the requirements hierarchy issues.