Support for AOT processing with GraalVM tracing agent is not consistent #30511

sbrannen · 2023-05-16T09:02:03Z

Overview

In Spring Framework 6.0.9, we made changes to the Spring TestContext Framework (TCF) to allow AOT processing with the GraalVM tracing agent and Native Build Tools.

Specifically, we introduced a TestAotDetector utility that is specific to the TCF. This detector considers the current runtime to be in "AOT runtime mode" if the spring.aot.enabled Spring property is set to true or the GraalVM org.graalvm.nativeimage.imagecode JVM system property is set to any non-empty value other than agent.

Since Spring Boot's testing support uses AotDetector.useGeneratedArtifacts() in various places, the Boot team should investigate whether Spring Boot Test should migrate from AotDetector to TestAotDetector.

Related Issues

Support Test AOT processing with GraalVM tracing agent and Native Build Tools #30281

The text was updated successfully, but these errors were encountered:

mhalbritter · 2023-05-16T12:37:24Z

Isn't there a more general problem? When using the agent with bootRun, this fails too. Using -Pagent, which enables the GraalVM agent, sets the environment variable org.graalvm.nativeimage.imagecode to agent. The AotDetector.useGeneratedArtifacts() then returns true because NativeDetector.inNativeImage returns true, because it only checks org.graalvm.nativeimage.imagecode for null (and does not ignore agent, like the TestAotDetector does).

Say i don't use AOT processing, then try to use -Pagent bootRun. We will now think we run in a native image (NativeDetector.inNativeImage returned true) and try to load classes which aren't there (the AOT processed ApplicationContextInitializer).

Shouldn't the NativeDetector be changed to ignore agent?

mhalbritter · 2023-05-16T12:40:57Z

If I run -Pagent test, I get this stacktrace while running processTestAot:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::       (v3.0.7-SNAPSHOT)

2023-05-16T14:38:10.409+02:00 ERROR 17669 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalArgumentException: Could not find class [com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:334) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.instantiateInitializer(AotApplicationContextInitializer.java:80) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.initialize(AotApplicationContextInitializer.java:71) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.lambda$forInitializerClasses$0(AotApplicationContextInitializer.java:61) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:605) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:385) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:309) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1388) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:545) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContextForAotProcessing(SpringBootContextLoader.java:113) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:263) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:232) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.lambda$processAheadOfTime$4(TestContextAotGenerator.java:204) ~[spring-test-6.0.9.jar:6.0.9]
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[na:na]
        at org.springframework.util.MultiValueMapAdapter.forEach(MultiValueMapAdapter.java:179) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:196) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:158) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.performAotProcessing(TestAotProcessor.java:91) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:72) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:39) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AbstractAotProcessor.process(AbstractAotProcessor.java:82) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.test.context.SpringBootTestAotProcessor.main(SpringBootTestAotProcessor.java:63) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
Caused by: java.lang.ClassNotFoundException: com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
        at java.base/java.lang.Class.forName0(Native Method) ~[na:na]
        at java.base/java.lang.Class.forName(Class.java:467) ~[na:na]
        at org.springframework.util.ClassUtils.forName(ClassUtils.java:284) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:324) ~[spring-core-6.0.9.jar:6.0.9]
        ... 25 common frames omitted

2023-05-16T14:38:10.419+02:00  WARN 17669 --- [           main] o.s.t.c.aot.TestContextAotGenerator      : Failed to generate AOT artifacts for test classes [com.lingh.AddRemoveDatasourceTest]

org.springframework.test.context.aot.TestContextAotException: Failed to load ApplicationContext for AOT processing for test class [com.lingh.AddRemoveDatasourceTest]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:272) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:232) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.lambda$processAheadOfTime$4(TestContextAotGenerator.java:204) ~[spring-test-6.0.9.jar:6.0.9]
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[na:na]
        at org.springframework.util.MultiValueMapAdapter.forEach(MultiValueMapAdapter.java:179) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:196) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:158) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.performAotProcessing(TestAotProcessor.java:91) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:72) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:39) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AbstractAotProcessor.process(AbstractAotProcessor.java:82) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.test.context.SpringBootTestAotProcessor.main(SpringBootTestAotProcessor.java:63) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
Caused by: java.lang.IllegalArgumentException: Could not find class [com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:334) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.instantiateInitializer(AotApplicationContextInitializer.java:80) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.initialize(AotApplicationContextInitializer.java:71) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.lambda$forInitializerClasses$0(AotApplicationContextInitializer.java:61) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:605) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:385) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:309) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1388) ~[spring-boot-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:545) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContextForAotProcessing(SpringBootContextLoader.java:113) ~[spring-boot-test-3.0.7-SNAPSHOT.jar:3.0.7-SNAPSHOT]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:263) ~[spring-test-6.0.9.jar:6.0.9]
        ... 11 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
        at java.base/java.lang.Class.forName0(Native Method) ~[na:na]
        at java.base/java.lang.Class.forName(Class.java:467) ~[na:na]
        at org.springframework.util.ClassUtils.forName(ClassUtils.java:284) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:324) ~[spring-core-6.0.9.jar:6.0.9]
        ... 25 common frames omitted

That's because org.springframework.boot.SpringApplication#addAotGeneratedInitializerIfNecessary (which is not in test support code!) has been called and it thinks it runs in a native image.

sbrannen · 2023-05-16T12:58:14Z

Shouldn't the NativeDetector be changed to ignore agent?

That's a very good question, @mhalbritter, and it might be the best solution for the time being.

I've been advocating since early 2020 that applications and build tools need a first-class mechanism for determining when an application is running in the JVM with the GraalVM tracing agent active, but it hasn't gained much traction from the GraalVM team.

see Support agent as org.graalvm.nativeimage.imagecode system property value oracle/graal#2395

In my last comment in that issue, I stated the following.

please have the agent set a different system property that can be queried in order to determine if code is running on the JVM with the agent enabled.

Lately, I believe that would be the better choice for the community. The org.graalvm.nativeimage.imagecode system property should probably remain reserved for use with the buildtime and runtime values.

@sdeleuze, @bclozel, @wilkinsona, @philwebb, @jhoeller, thoughts?

wilkinsona · 2023-05-16T13:08:48Z

Given the lack of traction on oracle/graal#2395, I think it makes sense to broaden our reliance on the NBT plugins setting org.graalvm.nativeimage.imagecode to agent. There doesn't appear to be any other path forward here that isn't blocked.

sdeleuze · 2023-05-16T13:12:16Z

Shouldn't the NativeDetector be changed to ignore agent?

The agent should detect the codepath used on native to generate the most accurate hints, so I don't think we should change NativeDetector behavior.

Maybe we should raise that point in the next meeting we have with the GraalVM team?

wilkinsona · 2023-05-16T13:16:09Z

I don't think we should change NativeDetector behavior.

But we could change AotDetector though? IMO, useGeneratedArtifacts() ought to return false when running with the agent.

sbrannen · 2023-05-16T13:22:21Z

I don't think we should change NativeDetector behavior.

But we could change AotDetector though? IMO, useGeneratedArtifacts() ought to return false when running with the agent.

Yes, exactly -- like what TestAotDetector does.

That's actually what I meant to say previously, but I accidentally replied to the suggestion for changing the NativeDetector. Sorry for the mix-up.

And that would make the TestAotDetector obsolete.

snicoll · 2023-05-21T08:27:05Z

I think that's what we should do, so I am moving this back to framework.

mhalbritter · 2023-05-22T11:38:13Z

The agent should detect the codepath used on native to generate the most accurate hints, so I don't think we should change NativeDetector behavior.

I'm not sure I agree to that line of reasoning. IMHO the NativeDetector should return true if (and only if) running a native image and AotDetector should return true if (and only if) running in AOT mode (which is always the case in native image).

When running my integration tests while not using AOT mode with the agent attached, I would assume that it will generate hints for all dynamic behavior (reflection dependency injection etc.) and that both NativeDetector and AotDetector return false.

If I want to generate hints with the agent for the AOT code path, then I would need to run the tests in AOT mode, too.

What's the idea behind NativeDetector or AotDetector return true when running with an attached agent? Better developer experience because users can run integration tests and only get hints for the AOT codepath out of the box?

sdeleuze · 2023-05-22T12:20:43Z

NativeDetector is on purpose pretty unopinionated and flexible, as soon as org.graalvm.nativeimage.imagecode is set it returns true, which means the app runs using one of the flavors of native image. In practice, it is set to buildtime or runtime by native-image compiler, or to agent by various tools that need to emulate native code path on the JVM.

This behavior looks correct to me as for some use cases, it is important to exercise the native code path on the JVM. The goal is not just to have more optimal hints, the goal is also to not break when running the app on native with tracing agent generated hints due to missing hints.

snicoll · 2023-05-22T13:23:01Z

This behavior looks correct to me as for some use cases, it is important to exercise the native code path on the JVM

Yes, but to be consistent with that statement, said code should run with AOT optimizations and it obviously can't. It looks wrong to me to require the native specific code path to run, while the most important bit, the one that avoids a bunch of code paths thanks to AOT, doesn't run. NativeDetector is obviously very generic on purpose but having it enabled while AOT is not (in the form of optimization, or execution) feels wrong to me.

This commit refines how native agent detection for both test and application executions. It rollbacks the introduction of TestAotDetector done in 1113096 and instead updates AotDetector.useGeneratedArtifacts() to only detect "buildtime" and "runtime" imagecode system property value which leverages a new method NativeDetector.inNativeImage(NativeDetector.Context...). Closes spring-projectsgh-30511

This commit refines how native agent detection works for both test and application executions. It rollbacks the introduction of TestAotDetector done in 1113096 and instead updates AotDetector.useGeneratedArtifacts() to only detect "buildtime" and "runtime" imagecode system property value which leverages a new method NativeDetector.inNativeImage(NativeDetector.Context...). Closes spring-projectsgh-30511

This commit refines how native agent detection works for both test and application executions. It rollbacks the introduction of TestAotDetector done in 1113096 and instead updates AotDetector.useGeneratedArtifacts() to only detect "buildtime" and "runtime" imagecode system property values by leveraging a new method NativeDetector.inNativeImage(NativeDetector.Context...). Closes spring-projectsgh-30511

This commit refines how native agent detection works for both test and application executions. It rolls back the introduction of TestAotDetector done in 1113096 and instead updates AotDetector.useGeneratedArtifacts() to only detect "buildtime" and "runtime" imagecode system property values by leveraging a new method NativeDetector.inNativeImage(NativeDetector.Context...). Closes spring-projectsgh-30511

This commit refines how native agent detection works for both test and application executions. It rolls back the introduction of TestAotDetector done in 1113096 and instead updates AotDetector.useGeneratedArtifacts() to only detect "buildtime" and "runtime" imagecode system property values by leveraging a new method NativeDetector.inNativeImage(NativeDetector.Context...). This commit also adds a workaround for oracle/graal#6691. Closes spring-projectsgh-30511

See gh-30511

This commit refines how GraalVM tracing agent detection works for both test and application executions. It rolls back the introduction of TestAotDetector done in 1113096 and instead updates AotDetector.useGeneratedArtifacts() to only detect "buildtime" and "runtime" imagecode system property values by leveraging a new method NativeDetector.inNativeImage(NativeDetector.Context...). This commit also adds a workaround for oracle/graal#6691. Closes spring-projectsgh-30511

See spring-projectsgh-30511

sbrannen mentioned this issue May 16, 2023

Support Test AOT processing with GraalVM tracing agent and Native Build Tools #30281

Closed

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label May 16, 2023

mhalbritter added type: bug A general bug theme: aot An issue related to Ahead-of-time processing and removed status: waiting-for-triage An issue we've not yet triaged or decided on labels May 16, 2023

snicoll transferred this issue from spring-projects/spring-boot May 21, 2023

snicoll changed the title ~~Investigate test AOT processing support with GraalVM tracing agent and Native Build Tools~~ Support AOT processing with GraalVM tracing agent is not consistent May 21, 2023

snicoll changed the title ~~Support AOT processing with GraalVM tracing agent is not consistent~~ Support for AOT processing with GraalVM tracing agent is not consistent May 21, 2023

snicoll added this to the 6.0.x milestone May 21, 2023

sdeleuze self-assigned this May 25, 2023

sdeleuze modified the milestones: 6.0.x, 6.0.10 May 25, 2023

sdeleuze mentioned this issue May 26, 2023

[GR-46741] NoClassDefFoundError thrown on the JVM but not with native image oracle/graal#6691

Closed

sdeleuze closed this as completed in 1cf6d1d May 26, 2023

jhoeller added a commit that referenced this issue Jun 5, 2023

Shorten constant names to BUILD and RUN (for consistent spelling)

22ef48b

See gh-30511

mdeinum pushed a commit to mdeinum/spring-framework that referenced this issue Jun 29, 2023

Shorten constant names to BUILD and RUN (for consistent spelling)

fc94f88

See spring-projectsgh-30511

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for AOT processing with GraalVM tracing agent is not consistent #30511

Support for AOT processing with GraalVM tracing agent is not consistent #30511

sbrannen commented May 16, 2023 •

edited

mhalbritter commented May 16, 2023 •

edited

mhalbritter commented May 16, 2023 •

edited

sbrannen commented May 16, 2023

wilkinsona commented May 16, 2023 •

edited

sdeleuze commented May 16, 2023 •

edited

wilkinsona commented May 16, 2023

sbrannen commented May 16, 2023

snicoll commented May 21, 2023

mhalbritter commented May 22, 2023 •

edited

sdeleuze commented May 22, 2023

snicoll commented May 22, 2023

Support for AOT processing with GraalVM tracing agent is not consistent #30511

Support for AOT processing with GraalVM tracing agent is not consistent #30511

Comments

sbrannen commented May 16, 2023 • edited

Overview

Related Issues

mhalbritter commented May 16, 2023 • edited

mhalbritter commented May 16, 2023 • edited

sbrannen commented May 16, 2023

wilkinsona commented May 16, 2023 • edited

sdeleuze commented May 16, 2023 • edited

wilkinsona commented May 16, 2023

sbrannen commented May 16, 2023

snicoll commented May 21, 2023

mhalbritter commented May 22, 2023 • edited

sdeleuze commented May 22, 2023

snicoll commented May 22, 2023

sbrannen commented May 16, 2023 •

edited

mhalbritter commented May 16, 2023 •

edited

mhalbritter commented May 16, 2023 •

edited

wilkinsona commented May 16, 2023 •

edited

sdeleuze commented May 16, 2023 •

edited

mhalbritter commented May 22, 2023 •

edited