CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot) #42

michaelkrog · 2018-03-01T14:53:32Z

I am having an issue wen running my application with QueryDSL on OpenJDK.

I have 2 classes that I generate QueryDsl classes for. One(Project) is a MongoEntity class that I annotated with @QueryEntity and the other(Service) is a POJO also with @QueryEntity.

I use com.querydsl.apt.QuerydslAnnotationProcessor for generating the classes.

When I compile and run it locally everything works fine. This is using the following JDK:


java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

But when built for production using this JDK:

java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

...and then run in a container with this JDK:

openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

...then I get this exception for the Service class, but not the Project class:

com.mysema.codegen.CodegenException: Compilation of public class Q_2124358794_1275614662_1275614662_1195259493 {

    public static Iterable<dk.codezoo.meerkat.model.Service> eval(Iterable<dk.codezoo.meerkat.model.Service> service_, String a1) {
java.util.List<dk.codezoo.meerkat.model.Service> rv = new java.util.ArrayList<dk.codezoo.meerkat.model.Service>();
for (dk.codezoo.meerkat.model.Service service : service_) {
    try {
        if (service.getName().toLowerCase().contains(a1)) {
            rv.add(service);
        }
    } catch (NullPointerException npe) { }
}
return rv;    }

}

 failed.
warning: Supported source version 'RELEASE_6' from annotation processor 'org.jvnet.hudson.annotation_indexer.AnnotationProcessorImpl' less than -source '1.8'
/Q_2124358794_1275614662_1275614662_1195259493.java:3: error: package dk.codezoo.meerkat.model does not exist
    public static Iterable<dk.codezoo.meerkat.model.Service> eval(Iterable<dk.codezoo.meerkat.model.Service> service_, String a1) {
                                                                                                   ^
/Q_2124358794_1275614662_1275614662_1195259493.java:3: error: package dk.codezoo.meerkat.model does not exist
    public static Iterable<dk.codezoo.meerkat.model.Service> eval(Iterable<dk.codezoo.meerkat.model.Service> service_, String a1) {
                                                   ^
2 errors
1 warning

	at com.mysema.codegen.JDKEvaluatorFactory.compile(JDKEvaluatorFactory.java:74)
	at com.mysema.codegen.AbstractEvaluatorFactory.createEvaluator(AbstractEvaluatorFactory.java:128)
	at com.querydsl.collections.DefaultEvaluatorFactory.createEvaluator(DefaultEvaluatorFactory.java:151)
	at com.querydsl.collections.DefaultQueryEngine.evaluateSingleSource(DefaultQueryEngine.java:166)
	at com.querydsl.collections.DefaultQueryEngine.list(DefaultQueryEngine.java:80)
	at com.querydsl.collections.AbstractCollQuery.fetch(AbstractCollQuery.java:182)
	at dk.codezoo.meerkat.orchestration.DockerOrchestrationClient$DockerServiceClient.findAll(DockerOrchestrationClient.java:108)
	at dk.codezoo.meerkat.service.ServiceService.findAll(ServiceService.java:64)
	at dk.codezoo.meerkat.controller.ServiceController.list(ServiceController.java:62)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.oauth2.provider.authentication.OAuth2AuthenticationProcessingFilter.doFilter(OAuth2AuthenticationProcessingFilter.java:176)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

The text was updated successfully, but these errors were encountered:

Shredder121 · 2018-03-01T15:02:29Z

package dk.codezoo.meerkat.model does not exist

Can you check this? Is it only happening with those JDKs?

michaelkrog · 2018-03-01T15:08:20Z

That package does exist. I have tried to unpack the jar just to check it, and it is there.
The two classes are both in the package. One works the other does not.

I just tried to force our build process to update the docker image it uses(openjdk:8-jdk), and that changed the JDK to this:

openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

But to no avail. It fails with the same exception.

~~But I am only seeing it when I run it on OpenJDK.~~

michaelkrog · 2018-03-01T15:17:19Z

Okay. I just extracted the Jar built for production and ran that locally... And it fails...
So its not only OpenJDK.

Weird. If I built it with maven locally on 1.8.0_121(mac) and run it locally, there is not issue. If built it with 1.8.0_161(linux) and I run that locally, then it fails.

michaelkrog · 2018-03-02T08:00:33Z

I upgraded my mac to JDK 1.8.0_161 and still I am unable to reproduce it when I build i locally.

I then tried to login to our build server(JDK 1.8.0_161 linux), checked out the code, did a fresh build(mvn clean install) and copied the jar-file to my local machine and it fails.

So weird... I'll try to setup a virtual linux locally and reproduce it there.

Shredder121 · 2018-03-02T08:14:17Z

Yes please do, I can't fathom where it would go wrong but can go right in some cases.
So I'd like to hear any progress!

michaelkrog · 2018-03-02T09:03:14Z

I think I found the source of the issue: classpath.

This is the start of the resolved classpath when I run the project via spring-boot:run from intellij:

/Users/michael/Development/meerkat/meerkat/target/classes:/Users/michael/.m2/repository/org/springframework/boot/spring-boot-starter-web/1.5.3.RELEASE/spring-boot-starter-web-1.5.3.RELEASE.jar:/Users/michael/.m2/repository/org/springframework/boot/spring-boot-starter/1.5.3.RELEASE/spring-boot-starter-1.5.3.RELEASE.jar:/...

And this is the start of the classpath when I run the final artifact locally from the commandline (java -jar meerkat.jar):

/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes!:/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/lib/spring-boot-starter-web-1.5.3.RELEASE.jar!:/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/lib/spring-boot-starter-1.5.3.RELEASE.jar!:/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/lib/spring-boot-1.5.3.RELEASE.jar!:...

Weird classpath

/Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes! looks a bit weird, doesn't it?

michaelkrog · 2018-03-02T09:21:15Z

It comes down to

codegen/src/main/java/com/mysema/codegen/SimpleCompiler.java

Lines 71 to 72 in 60b65f9

    
           String decodedPath = URLDecoder.decode(url.getPath(), "UTF-8"); 
        
           paths.add(new File(decodedPath).getAbsolutePath());

The first URL retrieved in my case is jar:file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes!/

At line 71 it is being decoded to file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes!/

At line 72 the absolute path becomes /Users/michael/Development/meerkat/meerkat/file:/Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/classes!

michaelkrog · 2018-03-06T07:22:18Z

I am unable to make it work.
SimpleCompiler.getClassPath(parent); at https://github.com/querydsl/codegen/blob/master/src/main/java/com/mysema/codegen/JDKEvaluatorFactory.java#L57 returns a classpath that does not work when the application is run as an executable jar built with Spring Boot (https://docs.spring.io/spring-boot/docs/current/reference/html/build-tool-plugins-maven-plugin.html).

The classloader is a TomcatEmbeddedWebappClassLoader which resolves jars in the executable jar(Fx. /Users/michael/Development/meerkat/meerkat.jar!/BOOT-INF/lib/spring-boot-starter-tomcat-1.5.3.RELEASE.jar!). But the classpath returned from SimpleCompiler is then broken and classes can then not be resolved.

My knowledge about JavaCompiler and the codegen project is to sparse. I have been unable to create a fix.

Shredder121 · 2018-03-06T07:49:33Z

Yes the issue is known, I'll continue with my fix

Shredder121 · 2018-03-06T07:50:00Z

Thanks for looking into it though, much appreciated!

matteo-gallo-bb · 2018-03-27T15:22:06Z

@Shredder121 I just created a PR to add the support for Codegen to generate code when running in a fat JAR. It should fix this bug and others related to this.
I'm using this fix and it works for me with Spring Boot, Spring Data Key Value, spring-data-hazelcast and QueryDsl Collections.
The PR is: #43
If you can, please review it and let me know if there are possible improvements to do.

michaelkrog · 2018-06-15T10:16:29Z

This issue is indeed fixed by #43.

michaelkrog · 2021-01-21T10:47:47Z

#43 was never accepted even after this many years.

Is there any solution in sight or is building a custom jar still the only viable solution in order to use this with spring boot?

viktorzub · 2021-07-08T11:31:31Z

Any updates here?

viktorzub · 2021-07-12T10:38:58Z

@michaelkrog do you how to build custom jar to make it work on ubuntu/macos?

michaelkrog changed the title ~~CodegenException when run in OpenJDK~~ CodegenException when built by 1.8.0_161 Mar 1, 2018

michaelkrog changed the title ~~CodegenException when built by 1.8.0_161~~ CodegenException mystery Mar 2, 2018

michaelkrog changed the title ~~CodegenException mystery~~ CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot) Mar 2, 2018

matteo-gallo-bb mentioned this issue Mar 27, 2018

Code generation support for application running in a fat JAR #43

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot) #42

CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot) #42

michaelkrog commented Mar 1, 2018

Shredder121 commented Mar 1, 2018

michaelkrog commented Mar 1, 2018 •

edited

michaelkrog commented Mar 1, 2018 •

edited

michaelkrog commented Mar 2, 2018 •

edited

Shredder121 commented Mar 2, 2018

michaelkrog commented Mar 2, 2018 •

edited

michaelkrog commented Mar 2, 2018 •

edited

michaelkrog commented Mar 6, 2018 •

edited

Shredder121 commented Mar 6, 2018

Shredder121 commented Mar 6, 2018

matteo-gallo-bb commented Mar 27, 2018

michaelkrog commented Jun 15, 2018

michaelkrog commented Jan 21, 2021

viktorzub commented Jul 8, 2021

viktorzub commented Jul 12, 2021

CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot) #42

CodegenException when using TomcatEmbeddedWebappClassLoader (fx. Spring Boot) #42

Comments

michaelkrog commented Mar 1, 2018

Shredder121 commented Mar 1, 2018

michaelkrog commented Mar 1, 2018 • edited

michaelkrog commented Mar 1, 2018 • edited

michaelkrog commented Mar 2, 2018 • edited

Shredder121 commented Mar 2, 2018

michaelkrog commented Mar 2, 2018 • edited

Weird classpath

michaelkrog commented Mar 2, 2018 • edited

michaelkrog commented Mar 6, 2018 • edited

Shredder121 commented Mar 6, 2018

Shredder121 commented Mar 6, 2018

matteo-gallo-bb commented Mar 27, 2018

michaelkrog commented Jun 15, 2018

michaelkrog commented Jan 21, 2021

viktorzub commented Jul 8, 2021

viktorzub commented Jul 12, 2021

michaelkrog commented Mar 1, 2018 •

edited

michaelkrog commented Mar 1, 2018 •

edited

michaelkrog commented Mar 2, 2018 •

edited

michaelkrog commented Mar 2, 2018 •

edited

michaelkrog commented Mar 2, 2018 •

edited

michaelkrog commented Mar 6, 2018 •

edited