gradle aggregate : mixed reports ?

[mmanach]

Hello,

Just to let you know of a strange behavior with gradle on Windows 10.

1. I run tests of project A from IntelliJ.
2. Then I check the serenity report, all ok
3. (I noticed that a java process is still here after this execution, seems to be the gradle daemon)
4. I run tests of project B from IntelliJ
5. Then I check the serenity report, and some part are mixed with the report of project A
6. To have a clean report, I need to kill this gradle daemon

[wakaleo]

This could be due to Windows file locks. Can you produce two projects that reproduce the issue?

[yemo]

I have the same problem when executing gradle in command line to run test. Not sure how 'aggregate' works and where it collects previous run project report files. two projects are in different folders, the report data source may come from memory.

[wakaleo]

The aggregate task collects the JSON test outcomes from the output directory (usually target/site/serenity) and converts them into an aggregate report. The only way I could imagine it generating test results from two projects is if the two projects produce results to the same target directory.

[yemo]

First time I seen this problem is on our Bamboo server. I don't think Bamboo agent check out two projects into same directory. I checked the Bamboo agent, two projects in different directories, and their outcomes in their own directories target folder.
It also happen on my local machine occasionally. After I ran one project, I ran the other project. Even I deleted the second project's target folder, and ran again, the first project info still showed on the second projects report (feature tab). It can be fixed by restart my local machine (kill gradle deamon), so I assumed aggregate task may also read some thing in the memory.

[andres-florit]

I experienced a different issue, but also produced by gradle daemon.

My issue was getting some previous erased capabilities or features in the generated report for the same project. I.e, if I had a feature A, then generated the report, then renamed the feature A to B and generated the report again, then I got both, A and B in some report tabs. Even if executed the clean task or removed target, build, and /.gradle folders manually. Something was cached somewhere and I found it in my ~/.gradle/daemon/3.5 folder. Removing that folder content before each test (besides executing clean task) or disabling gradle daemon worked for me.

To disable gradle daemon, add the following line to the ~/.gradle/gradle.properties

org.gradle.daemon=false

Environment: Ubuntu 16.04, Gradle 3.5, Java 1.8.0_101

To reproduce the issue, clone journey planner repo and play changing features and capabilities changing names and moving them in the hierarchy. Generate reports between changes. Use an installed gradle (not the wrapper) and local browser (no tested with selenium hub/grid). I think I experienced the same issue with the serenity cucumber starter repo too, didn't test it again if disabling the daemon fix it.

[wakaleo]

This sounds like a Gradle daemon issue rather than a Serenity one - the clean task should clear any caches that the Gradle daemon is using. I'm not sure what Serenity can do about it.