Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java 11 and AspectJ LTW #909

Closed
mjp91 opened this issue Jul 19, 2019 · 14 comments
Closed

Java 11 and AspectJ LTW #909

mjp91 opened this issue Jul 19, 2019 · 14 comments
Labels
declined: otherproject 👽 This issue should be reported to other project

Comments

@mjp91
Copy link

mjp91 commented Jul 19, 2019

After updating to JDK 11 the Maven Surefire plugin fails to run when both Jacoco agent and AspectJ weaver agent present.

Steps to reproduce

JaCoCo version: 0.8.4
Operating system: Windows 10
JDK: open-jdk-11.0.1
Tool integration: Maven
aspectjweaver: 1.9.4

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${maven-surefire-plugin.version}</version>
    <configuration>
        <argLine>
            @{argLine} -javaagent:${user.home}/.m2/repository/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar
        </argLine>
        <trimStackTrace>false</trimStackTrace>
    </configuration>
</plugin>
<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.4</version>
    <configuration>
        <skip>false</skip>
        <includes>
            ...
        </includes>
        <excludes>
            ...
        </excludes>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>report</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Expected behaviour

Tests to run as they did in JDK 8.

Actual behaviour

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
AspectJ Internal Error: unable to add stackmap attributes. null
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30.903 s
[INFO] Finished at: 2019-07-19T15:04:42+01:00
[INFO] Final Memory: 79M/280M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project ***: There are test failures.
[ERROR] 
[ERROR] Please refer to C:\Users\#\IdeaProjects\#\services\target\surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] There was an error in the forked process
[ERROR] Method "$jacocoData" in class #/function/bank/BankFeedRuleFunctionTest has illegal signature "Ljava/lang/Object;"
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR] Method "$jacocoData" in class #/function/bank/BankFeedRuleFunctionTest has illegal signature "Ljava/lang/Object;"
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:656)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:282)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
[ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
[ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
[ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
[ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
[ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
[ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
[ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
[ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
[ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
[ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[ERROR] at org.codehaus.classworlds.Launcher.main(Launcher.java:47)
@mjp91
Copy link
Author

mjp91 commented Jul 19, 2019

Seems to be resolved by using 0.8.3

@Godin
Copy link
Member

Godin commented Jul 19, 2019

Similarly to #896 (comment) :

Seems to be resolved by using 0.8.3

the only change in JaCoCo version 0.8.4 that can explain this - is usage of condy (https://openjdk.java.net/jeps/309) for Java 11+ classes introduced by #845. Therefore I'm pretty sure that version of AspectJ weaver agent that you use doesn't correctly/fully support Java 11 bytecode.

Looking only at the error message in stacktrace

Method "$jacocoData" in class #/function/bank/BankFeedRuleFunctionTest has illegal signature "Ljava/lang/Object;"

one can only say that $jacocoData generated by JaCoCo for Java 11+ classes is not method, but a name of dynamic constant aka condy with absolutely valid signature of type Ljava/lang/Object; - see https://github.com/jacoco/jacoco/blob/v0.8.4/org.jacoco.core/src/org/jacoco/core/internal/instr/CondyProbeArrayStrategy.java#L56-L57 Addition of complete reproducer might help to nail why/how AspectJ messes this.

In any case please report this to developers of AspectJ weaver agent.

@Godin Godin closed this as completed Jul 19, 2019
@Godin Godin added the declined: otherproject 👽 This issue should be reported to other project label Jul 19, 2019
@mjp91
Copy link
Author

mjp91 commented Jul 20, 2019

Thank you for your quick response.

Bug raised with AspectJ: https://bugs.eclipse.org/bugs/show_bug.cgi?id=549438

@kriegaex
Copy link

kriegaex commented Jun 9, 2021

FYI, if anyone else has the same problem in the future: This is neither a JaCoCo nor an AspectJ problem, but simply caused by the fact that aspects were applied too broadly during load-time weaving, AspectJ also weaving aspects into Surefire, JUnit and JaCoCo itself. Proper AspectJ configuration avoids this, as explained in eclipse-aspectj/aspectj#68 (comment) and proven in https://github.com/kriegaex/AJ_LTWJacocoWeavingProblem_549438. That it worked before in JDK 8 was pure luck.

@ibragfir
Copy link

ibragfir commented Jan 3, 2022

@Godin @kriegaex tried your options but still doesn't work with Java 17. My aspects applied only within my app packages and i can confirm that from the .dumpstream files.
AspectJ: 1.9.8.RC3 (tried 1.9.7 as well)
JDK: Zulu 17.0.1
OS: Win 10
Jacoco: 0.8.7

Works fine though if I use https://github.com/bazelbuild/bazel/tree/master/third_party/java/jacoco version instead.
https://github.com/bazelbuild/bazel/blob/master/third_party/java/jacoco/0001-Disable-use-of-constant-dynamic-bytecode.patch

@kriegaex
Copy link

kriegaex commented Jan 4, 2022

@ibragfir, please do not hijack this closed issue. Firstly, it was created in July 2019 when the most recent Java release was 12. The LTS was 11, and this question also is about Java 11. Like I have proven in my answer, the issue was simply wrong AspectJ load-time weaving configuration and neither a JaCoCo problem nor an AspectJ bug.

I am not actually a JaCoCo user and only was involved here because AspectJ was in the game. But if you read the JaCoCo 0.8.7 release notes, you will see that Java 17 support is marked as "experimental" there. That using another JaCoCo build with a special patch works for you, indicates that there is an actual JaCoCo problem with Java 17. This should be dealt with in a separate issue, not in this one, because the two topics are unrelated. I suggest you create one (please first check if one exists already), add an MCVE and feel free to link back to your comment here where the story started accidentally. Thank you.

Besides, if you want to use the AspectJ compiler with Java 17, 1.9.8.RC3 is the correct version. 1.9.7 knows nothing about Java 17. For LTW that might still work, if there are not bytecode constructs unknown to the aspect weaver. But to be on the safe side, stick to 1.9.8.RC3.

@ibragfir
Copy link

ibragfir commented Jan 4, 2022

@kriegaex thanks. I don't see a problem with aspects being applied too broadly in this case. Does it explain the underlying issue?
I commented here because that patch solved the issue for me with Java 11 and Jacoco 0.8.4 as well and I believe this is the same problem that was introduced with Jacoco 0.8.4 and might be helpful for others. Thanks for your reply and recommendations.

@kriegaex
Copy link

kriegaex commented Jan 4, 2022

It explains the issue as described here. You are, according to your own words, using my solution for the issue, experiencing a completely different problem solved by upgrading JaCoCo. Therefore, you are off topic here and should open a new issue for your problem, like I said. If you disagree, publish an MCVE proving your point.

@ibragfir
Copy link

ibragfir commented Jan 4, 2022

I'm experiencing exactly the same issue with Java 11 and Java 17 which can be solved by using that patched version of JaCoCo.
I can't publish the code of the company here but I have checked out your example and in that you're not actually reproducing the same issue.

The author here raised this problem which is exactly the one I'm trying to find a solution for.

AspectJ Internal Error: unable to add stackmap attributes. null
Method "$jacocoData" in class #/function/bank/BankFeedRuleFunctionTest has illegal signature "Ljava/lang/Object;"
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR] Method "$jacocoData" in class #/function/bank/BankFeedRuleFunctionTest has illegal signature "Ljava/lang/Object;"

And I don't think it was pure luck that it was working with Java 8 but the fact that the author upgraded from 0.8.3 to 0.8.4 along with Java 11. He wouldn't be able to use 0.8.4 with Java 8 since JaCoCo 0.8.4 is using Java 11+ feature (https://openjdk.java.net/jeps/309).

@ibragfir
Copy link

ibragfir commented Jan 4, 2022

Actually he could and worked because there's a condition not to use dynamic constants if the version is not >=11

@kriegaex
Copy link

kriegaex commented Jan 5, 2022

I'm experiencing exactly the same issue with Java 11 and Java 17

Why did you not say that right from the beginning? You only talked about Java 17 and different JaCoCo and AspectJ versions than the ones in this issue.

I can't publish the code of the company here

That is a lame excuse, unworthy of a developer. I said MCVE, not "original code". Simply fork my repository and adjust it until it reproduces the problem. Basically, your claim is "it does not work the way @kriegaex suggested" without even a scintilla of proof. I cannot debug what I cannot see. Can you?

I have checked out your example and in that you're not actually reproducing the same issue.

That might be true, my solution is still to be applied - maybe in addition to whatever fix there might be necessary in JaCoCo - in order to avoid instrumenting the bytecode of tools like JaCoCo, JUnit and Surefire when running tests including aspects which apply too broadly.

@kriegaex
Copy link

kriegaex commented Jan 5, 2022

I can't publish the code of the company here

That is a lame excuse, unworthy of a developer. I said MCVE, not "original code". Simply fork my repository and adjust it until it reproduces the problem.

OK, I just did that for you, see my latest commits. I managed to reproduce the problem now, despite excluding foreign packages, by adding something using virtual methods (a Shape interface and a class Circle implementing it), then adding a test for same. My original class under test was simpler than that.

It is actually #1151, which also the patch you mentioned relates to. See also JEP 309. I built the current JaCoCo snapshot locally without the patch, and it still reproduces the problem. With the patch, like you said, the problem goes away. Now we have something reproducible to look into for AspectJ.

@kriegaex
Copy link

kriegaex commented Jan 5, 2022

OK, the solution is actually very simple, see eclipse-aspectj/aspectj#68 (comment). Quote:

When running Maven with debug logging, I see (with added line breaks and comments):

[DEBUG] Forking command line: cmd.exe /X /C "
  "C:\Program Files\Java\jdk-17\bin\java"

    # JaCoCo agent
    -javaagent:C:\\Users\\alexa\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.7\\org.jacoco.agent-0.8.7-runtime.jar=destfile=C:\\Users\\alexa\\Documents\\java-src\\AJ_LTWJacocoWeavingProblem_549438\\target\\jacoco.exec

    # AspectJ weaver needs access to java.base/java.lang on JDK 16+
    --add-opens java.base/java.lang=ALL-UNNAMED

    # AspectJ weaving agent
    -javaagent:C:\Users\alexa/.m2/repository/org/aspectj/aspectjweaver/1.9.8.RC3/aspectjweaver-1.9.8.RC3.jar

    # Maven Surefire executable booter
    -jar C:\Users\alexa\AppData\Local\Temp\surefire15834133402369997443\surefirebooter962938972777004100.jar

      # Maven Surefire parameters
      C:\Users\alexa\AppData\Local\Temp\surefire15834133402369997443
      2022-01-05T09-56-23_778-jvmRun1
      surefire11151023191849343689tmp
      surefire_08147447903974074624tmp
"

I.e., the JaCoCo agent is applied first and then the AspectJ weaver needs to deal with JaCoCo-instrumented byte code, whereas it should be the other way around: JaCoCo should create a coverage report for AspectJ-enhanced classes. So I added commit 487552b8, simply reversing the order of agents on the Surefire command line. This fixes the problem.


Bottom line: The order of Java agents simply was wrong. JaCoCo has to be applied after AspectJ. I had not thought of that before, but should have suggested it in the first place, in addition to limiting the aspect scope, which is still necessary.

Incorrect Surefire configuration:

<argLine>
  --add-opens java.base/java.lang=ALL-UNNAMED
  @{argLine}
  -javaagent:${user.home}/.m2/repository/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar
</argLine>

Correct Surefire configuration:

<argLine>
  --add-opens java.base/java.lang=ALL-UNNAMED
  -javaagent:${user.home}/.m2/repository/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar
  @{argLine}
</argLine>

@kriegaex
Copy link

kriegaex commented Aug 13, 2022

OK, reversing the order of instrumentation agents is suboptimal, because it might lead to incorrect coverage data, applying JaCoCo instrumentation to byte code no longer corresponding to the source code it was compiled from, now also containing aspect code. Today I found a better solution when re-investigating a related problem, see eclipse-aspectj/aspectj#170 (comment): use the -XnoInline AspectJ compiler or weaver option. This even enables you to get coverage data for the aspects themselves, if they are also compiled in the same module and have been instrumented by JaCoCo. This solution works in the current AspectJ version 1.9.9.1 and also back to 1.9.8 which fixed an old condy bug. In 1.9.9.2 (or 1.9.10, whatever comes first, a maintenance release or the Java 19 one), I will also improve the still incomplete AspectJ condy support some more, but for JaCoCo that is not necessary.

If a JaCoCo maintainer like @Godin is listening here: The fact that JaCoCo uses the exact same dynamic constant name and initialisation method name in each instrumented class is what causes problem with byte code engineering tools inlining code from class A into class B due to name clashes. Unique names like $jacocoData_[something_unique] and $jacocoInit_[something_unique] would help in this case. AspectJ would not be the only tool profiting from this enhancement. I remember that Byte Buddy advice code can also be inlined. That should lead to similar problems, if the BB code is covered by JaCoCo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
declined: otherproject 👽 This issue should be reported to other project
Projects
None yet
Development

No branches or pull requests

4 participants