-
-
Notifications
You must be signed in to change notification settings - Fork 513
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
serenity-junit5 dependency will break any tests execution (unit and integration) if jacoco plugin for code coverage is used in project #2887
Comments
Have you tried making sure Jacoco does not attempt to instrument the Serenity test classes? |
I did. I do not fully understand how instrumenting works, so cannot 100% guarantee, however, I've made these checks. On a compiled project executing The important note here is that The sure thing may be the issue is with jacoco itself, and this is how it works. I don't have enough knowledge unfortunately detect an exact root cause. |
I had a quick look, but really have absolutely no idea what it could be caused by - possibly some incompatibility between byte-buddy and jacoco. |
I took a look a day ago and I guess it is due to the JDK source/target version 16 being used in the When I ran using JDK 16 using source/target version 16, I got the following error log.
The error seems to arise in the I will spend some more time on this today and post my findings. |
I was able to bypass this issue in JDK 16 after excluding org.junit.jupiter package from jacoco instrumentation using the
My analysis is this:
Also found this Stackoverflow answer which seems to have an opinion against doing this in our @wakaleo is the functionality of the static block to instrument the |
Looks like it is an incompatibility between the jacoco instrumentation and the bytecode instrumentation that Serenity uses - I suspect the code in the static block is necessary (I didn't write this code myself so I'm not 100% sure). Do you have the link to the stackoverflow answer? I think your solution of bypassing jacoco instrumentation of the jupiter classes makes sense (no reason why they should be instrumented). |
@wakaleo This is the stackoverflow answer. Missed to link it in my earlier reply. Few more points after some more analysis:
|
I can't see using the -javaagent option as being a viable long-term solution, except as a workaround for jacoco usage, so I think the static code will need to stay as it is. @cliviu do you recall why we need to instrument the Assertions class in https://github.com/serenity-bdd/serenity-core/blob/main/serenity-junit5/src/main/java/net/serenitybdd/junit5/SerenityTestExecutionListener.java#L46? |
The work-around suggested by @vivganes seems to be the best approach to getting this working - the issue seems to be that jacoco is instrumenting test framework code, which it probably shouldn't be doing, and Serenity also needs to instrument the JUnit Assertions class to ensure that the reporting works correctly.
|
@wakaleo @vivganes thank you for looking into the issue. The suggested workaround resolves the case. I have double-checked that with both a synthetic simplified project and the actual Even though this behavior is not obvious (since Serenity JUnit4 did not have this issue), the workaround is totally acceptable. |
When there are both serenity-junit5 and jacoco-maven-plugin any test run will fail, even if there are no Serenity JUnit5 tests at all. It's impossible to workaround. Any project that uses jacoco code coverage tool cannot contain serenity-junit5.
Preconditions:
Checkout modified Serenity Example https://github.com/beyond-danube/serenity-junit-starter-junit5-jacoco-issue
Example contains:
Steps to reproduce:
mvn clean verify
(all good at this point, integration tests use JUnit4, unit tests use JUnit5, both serenity and jacoco reports are generated fine)pom.xml
uncommentserenity-junit5
dependencymvn clean verify
againActual Result:
The whole thing will fail on the test stage with
org.junit.platform.launcher.TestExecutionListener: Provider net.serenitybdd.junit5.SerenityTestExecutionListener could not be instantiated
even if Serenity JUnit5 were not used anywhere.Expected Result:
Dependency itself should not affect unrelated test execution.
The text was updated successfully, but these errors were encountered: