Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Having trouble setting up custom Weld on jetty12 #10150

Open
andresluuk opened this issue Jul 26, 2023 · 20 comments
Open

Having trouble setting up custom Weld on jetty12 #10150

andresluuk opened this issue Jul 26, 2023 · 20 comments
Labels
Question

Comments

@andresluuk
Copy link

Latest jetty12 build with ee9 or ee10

JDK 17.

I can see that jetty12 has CDI modules and we probably could enable those to get jettys internal weld CDI working, but for the purpose for this question I would no enable those modules.

I have a standalone weld 4 war (but I had the same issue also with weld 5.1), that is working in jetty11. The war contains all the files and APIs needed for weld. Looking at code I can see that it uses ServiceLoader (from jakarta.enterprise.inject.spi.CDI) to find out it's provided CDI implementation.
I can see that the service interface is loaded from the application:
'jar:file:/C:/work/servers/jetty-home-11.0.12/webapps/exp-weld40-beanClassAdd/WEB-INF/lib/jakarta.enterprise.cdi-api-3.0.0.jar!/jakarta/enterprise/inject/spi/CDIProvider.class'. And now it finds the implementation from WEB-INF/lib/weld-servlet-core-4.0.3.Final.jar
Now when I deployed the same application on weld12ee9 (also the same on weld12ee10) I can see that the interface is found from:
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.enterprise.cdi-api-3.0.1.jar!/jakarta/enterprise/inject/spi/CDIProvider.class And later on the implementation from weld-servlet-core-4.0.3.Final.jar is not found. The jar WEB-INF/lib/jakarta.enterprise.cdi-api-3.0.0.jar also exists, but CDIProvider is found from the server and for some reason, it does not find the implementation provided by the application.

I digged a little deeper and found out that the classpath has changed:
jetty11:

org.eclipse.jetty.start.Classpath$Loader@150fc7a7 URLs: [file:/C:/work/servers/jetty-home-11.0.12/resources/, 
file:/C:/work/servers/jetty-home-11.0.12/lib/logging/slf4j-api-2.0.0.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/logging/jetty-slf4j-impl-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-jakarta-servlet-api-5.0.2.jar,
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-http-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-server-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-xml-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-util-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-io-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-jndi-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-security-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-servlet-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-webapp-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-plus-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jakarta.transaction-api-2.0.0.jar,
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-annotations-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/annotations/asm-9.3.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/annotations/asm-analysis-9.3.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/annotations/asm-commons-9.3.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/annotations/asm-tree-9.3.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/annotations/jakarta.annotation-api-2.1.1.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/apache-jsp/org.eclipse.jdt.ecj-3.27.0.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/apache-jsp/org.eclipse.jetty.apache-jsp-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/apache-jsp/org.mortbay.jasper.apache-el-10.0.14.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/apache-jsp/org.mortbay.jasper.apache-jsp-10.0.14.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-client-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-alpn-client-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/jetty-deploy-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/websocket/websocket-core-common-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/websocket/websocket-core-client-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/websocket/websocket-core-server-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/websocket/websocket-servlet-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/websocket/jetty-jakarta-websocket-api-2.0.0.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/websocket/websocket-jakarta-client-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/websocket/websocket-jakarta-common-11.0.12.jar, 
file:/C:/work/servers/jetty-home-11.0.12/lib/websocket/websocket-jakarta-server-11.0.12.jar]

jetty12

2023-07-26 09:14:30.285 INFO  [0] [Core] java.net.URLClassLoader@893081e URLs: [
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-jakarta-servlet-api-5.0.2.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-nested-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-servlet-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-security-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-webapp-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-plus-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.transaction-api-2.0.1.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.interceptor-api-2.0.1.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.enterprise.cdi-api-3.0.1.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.inject-api-2.0.1.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-annotations-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/asm-9.5.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/asm-analysis-9.5.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/asm-commons-9.5.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/asm-tree-9.5.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/jakarta.annotation-api-2.0.0.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-apache-jsp/org.eclipse.jdt.ecj-3.27.0.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-apache-jsp/org.mortbay.jasper.apache-el-10.0.14.jar,
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-apache-jsp/org.mortbay.jasper.apache-jsp-10.0.14.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-apache-jsp-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-glassfish-jstl/jakarta.servlet.jsp.jstl.jakarta.servlet.jsp.jstl-api-2.0.0.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-glassfish-jstl/org.glassfish.web.jakarta.servlet.jsp.jstl-2.0.0.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-websocket-core-common-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-websocket-core-client-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-websocket-core-server-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-servlet-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-jakarta-websocket-api-2.0.0.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jakarta-client-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jakarta-common-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jakarta-server-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jetty-api-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jetty-common-12.0.0-SNAPSHOT.jar, 
file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jetty-server-12.0.0-SNAPSHOT.jar]

Jetty 11 did not have jakarta.enterprise.cdi-api-3.0.1.jar on server classpath. Is it there by design? I have not enabled any CDI modules in my knowledge:

Enabled Modules:
----------------
  0) resources                 ${jetty.base}\start.d\resources.ini
  1) logging/slf4j             transitive provider of logging/slf4j for logging-jetty
                               dynamic dependency of logging-jetty
  2) logging-jetty             transitive provider of logging for threadpool
                               transitive provider of logging for bytebufferpool
                               transitive provider of logging for server
  3) bytebufferpool            transitive provider of bytebufferpool for server
                               ini template available with --add-modules=bytebufferpool
  4) client                    ${jetty.base}\start.d\client.ini
  5) threadpool                transitive provider of threadpool for server
                               ini template available with --add-modules=threadpool
  6) server                    ${jetty.base}\start.d\server.ini
  7) deploy                    ${jetty.base}\start.d\deploy.ini
  8) sessions                  transitive provider of sessions for ee9-servlet
                               ini template available with --add-modules=sessions
  9) ee9-servlet               transitive provider of ee9-servlet for ee9-security
                               transitive provider of ee9-servlet for ee9-apache-jsp
                               transitive provider of ee9-servlet for ee9-webapp
                               transitive provider of ee9-servlet for ee9-jsp
 10) security                  transitive provider of security for ee9-security
 11) ee9-security              transitive provider of ee9-security for ee9-plus
                               transitive provider of ee9-security for ee9-webapp
 12) ee9-webapp                transitive provider of ee9-webapp for ee9-plus
                               transitive provider of ee9-webapp for ee9-deploy
                               ini template available with --add-modules=ee9-webapp
 13) jndi                      transitive provider of jndi for ee9-plus
 14) ee9-plus                  transitive provider of ee9-plus for ee9-annotations
 15) ee9-annotations           transitive provider of ee9-annotations for ee9-websocket-jakarta
                               transitive provider of ee9-annotations for ee9-apache-jsp
                               transitive provider of ee9-annotations for ee9-jsp
 16) ee9-apache-jsp            transitive provider of ee9-apache-jsp for ee9-jsp
 17) ee9-deploy                ${jetty.base}\start.d\ee9-deploy.ini
 18) ee9-glassfish-jstl        transitive provider of ee9-glassfish-jstl for ee9-jstl
 19) ee9-jsp                   ${jetty.base}\start.d\ee9-jsp.ini
 20) ee9-jstl                  ${jetty.base}\start.d\ee9-jstl.ini
 21) ee9-websocket-jakarta     ${jetty.base}\start.d\ee9-websocket-jakarta.ini
 22) http                      ${jetty.base}\start.d\http.ini

Are there any suggestions in order to get the service loader working again?
Maybe I can turn the application into child-first classloading somehow in order to get it working?
Can I remove the jar from servers classpath somehow?
Or if the interface is in the servers classpath, it should still find the implementation in the application..
@joakime
Copy link
Contributor

joakime commented Jul 26, 2023 •
edited

Server level CDI has always been problematic.
And fixing it has been very difficult as we ourselves never use CDI and do not know how to to fix it.
And when we reach out to the Jakarta CDI folks for help we get silence.

There was a time back in Jetty 9 where we attempted to get CDI working at the Server level properly, but it failed.
Then when Jetty 10 rolled around, we changed the modules & xml to simply enable CDI on a webapp if the webapp needed it. (No server level support)
We've kept that approach in Jetty 12, only enable CDI on a webapp.

As to how things are loaded with the Jetty CDI module, all lookups are done via the context's ClassLoader first.
But that ClassLoader, often a WebAppClassLoader, has it's own rules on where to look up things (following the servlet spec and various configurations within the WebAppClassLoader).

You can check the jetty server dump for details.

[my-base]$ java -jar /opt/jetty-home-12.0.0/start.jar jetty.server.dumpAfterStart=true

Look for the entries about Systemclasses and Serverclasses and those will indicate the rules being applied to the WebAppClassLoader.

Example:

|     |  +> Systemclasses Demo WebApp@21a5fd96 size=21
|     |  |  +> jakarta.
|     |  |  +> java.
|     |  |  +> javax.
|     |  |  +> org.eclipse.jetty.ee10.jaas.
|     |  |  +> org.eclipse.jetty.ee10.jsp.
|     |  |  +> org.eclipse.jetty.ee10.servlet.DefaultServlet
|     |  |  +> org.eclipse.jetty.ee10.servlet.NoJspServlet
|     |  |  +> org.eclipse.jetty.ee10.servlet.StatisticsServlet
|     |  |  +> org.eclipse.jetty.ee10.servlets.PushCacheFilter
|     |  |  +> org.eclipse.jetty.ee10.servlets.PushSessionCacheFilter
|     |  |  +> org.eclipse.jetty.ee10.websocket.jakarta.client.JakartaWebSocketClientContainerProvider
|     |  |  +> org.eclipse.jetty.ee10.websocket.jakarta.client.JakartaWebSocketShutdownContainer
|     |  |  +> org.eclipse.jetty.ee10.websocket.jakarta.server.config.
|     |  |  +> org.eclipse.jetty.ee10.websocket.server.
|     |  |  +> org.eclipse.jetty.ee10.websocket.servlet.
|     |  |  +> org.eclipse.jetty.jndi.
|     |  |  +> org.eclipse.jetty.security.jaas.
|     |  |  +> org.eclipse.jetty.websocket.api.
|     |  |  +> org.eclipse.jetty.websocket.server.
|     |  |  +> org.w3c.
|     |  |  +> org.xml.
|     |  +> Serverclasses Demo WebApp@21a5fd96 size=27
|     |  |  +> -org.eclipse.jetty.ee10.apache.
|     |  |  +> -org.eclipse.jetty.ee10.jaas.
|     |  |  +> -org.eclipse.jetty.ee10.jsp.
|     |  |  +> -org.eclipse.jetty.ee10.servlet.DefaultServlet
|     |  |  +> -org.eclipse.jetty.ee10.servlet.NoJspServlet
|     |  |  +> -org.eclipse.jetty.ee10.servlet.StatisticsServlet
|     |  |  +> -org.eclipse.jetty.ee10.servlet.listener.
|     |  |  +> -org.eclipse.jetty.ee10.servlets.
|     |  |  +> -org.eclipse.jetty.ee10.websocket.jakarta.client.JakartaWebSocketClientContainerProvider
|     |  |  +> -org.eclipse.jetty.ee10.websocket.jakarta.client.JakartaWebSocketShutdownContainer
|     |  |  +> -org.eclipse.jetty.ee10.websocket.jakarta.server.config.
|     |  |  +> -org.eclipse.jetty.ee10.websocket.server.
|     |  |  +> -org.eclipse.jetty.ee10.websocket.servlet.
|     |  |  +> -org.eclipse.jetty.jndi.
|     |  |  +> -org.eclipse.jetty.security.jaas.
|     |  |  +> -org.eclipse.jetty.websocket.api.
|     |  |  +> -org.eclipse.jetty.websocket.server.
|     |  |  +> file:///home/joakim/code/jetty/distros/bases/demos-12/lib/bouncycastle/
|     |  |  +> file:///home/joakim/code/jetty/distros/jetty-home-12.0.0/lib/logging/
|     |  |  +> org.eclipse.jdt.
|     |  |  +> org.eclipse.jetty.
|     |  |  +> org.eclipse.jetty.ee10.websocket.server.config.
|     |  |  +> org.eclipse.jetty.ee10.websocket.server.internal.
|     |  |  +> org.eclipse.jetty.logging.
|     |  |  +> org.eclipse.jetty.websocket.server.internal.
|     |  |  +> org.objectweb.asm.
|     |  |  +> org.slf4j.

I think the 3 lines ...

|     |  +> Systemclasses Demo WebApp@21a5fd96 size=21
|     |  |  +> jakarta.
|     |  |  +> java.
|     |  |  +> javax.

.. are impacting you.
Theses are from by the Servlet Spec, and are designed to prevent a webapp from replacing classes that start with those names with their own implementations.

@joakime
Copy link
Contributor

joakime commented Jul 26, 2023

In short, if you want your webapp's CDI to be used, then don't enable the server level CDI modules.

@andresluuk
Copy link
Author

I got the dump

C:\work\servers\jetty-home-12.0.0-SNAPSHOT>java  -jar start.jar jetty.base=base9 jetty.deploy.monitoredDir=../webapps jetty.server.dumpAfterStart=true
2023-07-26 15:10:10.115:INFO :oejs.Server:main: jetty-12.0.0-SNAPSHOT; built: 2023-07-26T05:17:51.528Z; git: fc872bacb25d9f9bfe25acc27a6d6f25b1d441b5; jvm 17.0.4+8
2023-07-26 15:10:10.138:INFO :oejdp.ScanningAppProvider:main: Deployment monitor ee9 in [file:///C:/work/servers/jetty-home-12.0.0-SNAPSHOT/webapps/] at intervals 1s
2023-07-26 15:10:10.145:INFO :oejs.DefaultSessionIdManager:main: Session workerName=node0
2023-07-26 15:10:10.181:INFO :oejs.AbstractConnector:main: Started ServerConnector@183ec003{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2023-07-26 15:10:10.209:INFO :oejs.Server:main: Started oejs.Server@16610890{STARTING}[12.0.0-SNAPSHOT,sto=5000] @2853ms
oejs.Server@16610890{STARTING}[12.0.0-SNAPSHOT,sto=5000] - STARTING
+= QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59924ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  +- org.eclipse.jetty.util.thread.ThreadPoolBudget@5965d37
|  += ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0} - STARTED
|  |  +> threads size=0
|  +> threads size=10
|     +> qtp1415157681-27 RUNNABLE tid=27 prio=5 SELECTING
|     +> qtp1415157681-26 RUNNABLE tid=26 prio=5 SELECTING
|     +> qtp1415157681-33 TIMED_WAITING tid=33 prio=5 IDLE
|     +> qtp1415157681-30 RUNNABLE tid=30 prio=5 SELECTING
|     +> qtp1415157681-32 RUNNABLE tid=32 prio=5 SELECTING
|     +> qtp1415157681-28-acceptor-0@3126dc4e-ServerConnector@183ec003{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} RUNNABLE tid=28 prio=3 ACCEPTING
|     +> qtp1415157681-31 RUNNABLE tid=31 prio=5 SELECTING
|     +> qtp1415157681-24 RUNNABLE tid=24 prio=5 SELECTING
|     +> qtp1415157681-29 RUNNABLE tid=29 prio=5 SELECTING
|     +> qtp1415157681-25 RUNNABLE tid=25 prio=5 SELECTING
+= oejut.ScheduledExecutorScheduler@51931956{STARTED} - STARTED
|  +> java.base@17.0.4/jdk.internal.misc.Unsafe.park(Native Method)
|  +> java.base@17.0.4/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
|  +> java.base@17.0.4/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
|  +> java.base@17.0.4/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
|  +> java.base@17.0.4/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
|  +> java.base@17.0.4/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
|  +> java.base@17.0.4/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
|  +> java.base@17.0.4/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
|  +> java.base@17.0.4/java.lang.Thread.run(Thread.java:833)
+- org.eclipse.jetty.io.ArrayByteBufferPool@3de8f619{min=0,max=65536,buckets=16,heap=0/1064828928,direct=0/1064828928}
|  +> direct size=16
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@eadd4fb{capacity=4096,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@740fb309{capacity=8192,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@7bd7d6d6{capacity=12288,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@43f02ef2{capacity=16384,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@239a307b{capacity=20480,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@2a8448fa{capacity=24576,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@6f204a1a{capacity=28672,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@2de56eb2{capacity=32768,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@5f8e8a9d{capacity=36864,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@5745ca0e{capacity=40960,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@3ad83a66{capacity=45056,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@3cce5371{capacity=49152,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@17bffc17{capacity=53248,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@6e535154{capacity=57344,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@15a34df2{capacity=61440,inuse=0(0%)}
|  |  +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@5b38c1ec{capacity=65536,inuse=0(0%)}
|  +> indirect size=16
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@338fc1d8{capacity=4096,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@4722ef0c{capacity=8192,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@48e1f6c7{capacity=12288,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@55cb6996{capacity=16384,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@1807e3f6{capacity=20480,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@480d3575{capacity=24576,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@f1da57d{capacity=28672,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@194fad1{capacity=32768,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@26abb146{capacity=36864,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@72c8e7b{capacity=40960,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@65f8f5ae{capacity=45056,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@431cd9b2{capacity=49152,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@38467116{capacity=53248,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@5b7a7f33{capacity=57344,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@790da477{capacity=61440,inuse=0(0%)}
|     +> org.eclipse.jetty.io.ArrayByteBufferPool$RetainedBucket@5c7933ad{capacity=65536,inuse=0(0%)}
+~ org.eclipse.jetty.util.resource.FileSystemPool@57bc27f5
+= oejsh.DefaultHandler@5fb759d6{showContext=true,favIcon=true,STARTED} - STARTED
+= oejsh.ContextHandlerCollection@4b8d604b{STARTED} - STARTED
+= oejd.DeploymentManager@5e7cd6cc{STARTED} - STARTED
|  += class org.eclipse.jetty.deploy.providers.ContextProvider@68c9d179[file:///C:/work/servers/jetty-home-12.0.0-SNAPSHOT/webapps/] - STARTED
|     +- ju.HashMap@0{size=0}
|     += oejur.ResourceFactoryInternals$LifeCycle@6e2829c7{STARTED} - STARTED
|     |  +> mounts size=0
|     +- org.eclipse.jetty.util.resource.ResourceFactory$2@3feb2dda
|     += oeju.Scanner@6a8658ff{STARTED} - STARTED
|        += oejut.ScheduledExecutorScheduler@1c742ed4{STARTED} - STARTED
|           +> java.base@17.0.4/jdk.internal.misc.Unsafe.park(Native Method)
|           +> java.base@17.0.4/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
|           +> java.base@17.0.4/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
|           +> java.base@17.0.4/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
|           +> java.base@17.0.4/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
|           +> java.base@17.0.4/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
|           +> java.base@17.0.4/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
|           +> java.base@17.0.4/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
|           +> java.base@17.0.4/java.lang.Thread.run(Thread.java:833)
+= oejs.DefaultSessionIdManager@333d4a8c{STARTED}[worker=node0] - STARTED
|  +? oejs.HouseKeeper@55de24cc{STARTED}[interval=600000, ownscheduler=false] - STARTED
+= ServerConnector@183ec003{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} - STARTED
|  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59908ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  +~ oejut.ScheduledExecutorScheduler@51931956{STARTED} - STARTED
|  +~ org.eclipse.jetty.io.ArrayByteBufferPool@3de8f619{min=0,max=65536,buckets=16,heap=0/1064828928,direct=0/1064828928}
|  += HttpConnectionFactory@1fa268de[HTTP/1.1] - STARTED
|  |  +- HttpConfiguration@aba625{32768/8192,8192/8192,https://:0,[]}
|  |     +> customizers size=0
|  |     +> formEncodedMethods size=2
|  |     |  +> POST
|  |     |  +> PUT
|  |     +> outputBufferSize=32768
|  |     +> outputAggregationSize=8192
|  |     +> requestHeaderSize=8192
|  |     +> responseHeaderSize=8192
|  |     +> headerCacheSize=1024
|  |     +> secureScheme=https
|  |     +> securePort=0
|  |     +> idleTimeout=-1
|  |     +> sendDateHeader=false
|  |     +> sendServerVersion=true
|  |     +> sendXPoweredBy=false
|  |     +> delayDispatchUntilContent=true
|  |     +> persistentConnectionsEnabled=true
|  |     +> maxErrorDispatches=10
|  |     +> minRequestDataRate=0
|  |     +> minResponseDataRate=0
|  |     +> requestCookieCompliance=RFC6265@7a6d7e92[INVALID_COOKIES, OPTIONAL_WHITE_SPACE, SPACE_IN_VALUES]
|  |     +> responseCookieCompliance=RFC6265@7a6d7e92[INVALID_COOKIES, OPTIONAL_WHITE_SPACE, SPACE_IN_VALUES]
|  |     +> notifyRemoteAsyncErrors=true
|  |     +> relativeRedirectAllowed=false
|  += SelectorManager@ServerConnector@183ec003{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} - STARTED
|  |  += oeji.ManagedSelector@16150369{STARTED} id=0 keys=0 selected=0 updates=0 - STARTED
|  |  |  += AdaptiveExecutionStrategy@6b09fb41/SelectorProducer@624ea235/PRODUCING/p=0/QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59903ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}][pc=0,pic=0,pec=0,epc=0]@2023-07-26T15:10:10.2324865+03:00 - STARTED
|  |  |  |  +- SelectorProducer@624ea235
|  |  |  |  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59903ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  |  |  +> updates @ 2023-07-26T15:10:10.2314942+03:00 size=0
|  |  |  +> keys @ 2023-07-26T15:10:10.2314942+03:00 size=0
|  |  += oeji.ManagedSelector@3932c79a{STARTED} id=1 keys=0 selected=0 updates=0 - STARTED
|  |  |  += AdaptiveExecutionStrategy@782859e/SelectorProducer@23f5b5dc/PRODUCING/p=0/QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59903ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}][pc=0,pic=0,pec=0,epc=0]@2023-07-26T15:10:10.2334843+03:00 - STARTED
|  |  |  |  +- SelectorProducer@23f5b5dc
|  |  |  |  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59902ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  |  |  +> updates @ 2023-07-26T15:10:10.2324865+03:00 size=0
|  |  |  +> keys @ 2023-07-26T15:10:10.2334843+03:00 size=0
|  |  += oeji.ManagedSelector@34bde49d{STARTED} id=2 keys=0 selected=0 updates=0 - STARTED
|  |  |  += AdaptiveExecutionStrategy@1b1cfb87/SelectorProducer@821330f/PRODUCING/p=0/QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59902ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}][pc=0,pic=0,pec=0,epc=0]@2023-07-26T15:10:10.2344803+03:00 - STARTED
|  |  |  |  +- SelectorProducer@821330f
|  |  |  |  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59901ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  |  |  +> updates @ 2023-07-26T15:10:10.2334843+03:00 size=0
|  |  |  +> keys @ 2023-07-26T15:10:10.2334843+03:00 size=0
|  |  += oeji.ManagedSelector@6f43c82{STARTED} id=3 keys=0 selected=0 updates=0 - STARTED
|  |  |  += AdaptiveExecutionStrategy@5db6b9cd/SelectorProducer@210ab13f/PRODUCING/p=0/QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59901ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}][pc=0,pic=0,pec=0,epc=0]@2023-07-26T15:10:10.2344803+03:00 - STARTED
|  |  |  |  +- SelectorProducer@210ab13f
|  |  |  |  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59901ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  |  |  +> updates @ 2023-07-26T15:10:10.2344803+03:00 size=0
|  |  |  +> keys @ 2023-07-26T15:10:10.2344803+03:00 size=0
|  |  += oeji.ManagedSelector@20b2475a{STARTED} id=4 keys=0 selected=0 updates=0 - STARTED
|  |  |  += AdaptiveExecutionStrategy@7857fe2/SelectorProducer@6f15d60e/PRODUCING/p=0/QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59900ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}][pc=0,pic=0,pec=0,epc=0]@2023-07-26T15:10:10.2354767+03:00 - STARTED
|  |  |  |  +- SelectorProducer@6f15d60e
|  |  |  |  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59900ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  |  |  +> updates @ 2023-07-26T15:10:10.2354767+03:00 size=0
|  |  |  +> keys @ 2023-07-26T15:10:10.2354767+03:00 size=0
|  |  += oeji.ManagedSelector@1be2019a{STARTED} id=5 keys=0 selected=0 updates=0 - STARTED
|  |  |  += AdaptiveExecutionStrategy@29d80d2b/SelectorProducer@58e1d9d/PRODUCING/p=0/QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59899ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}][pc=0,pic=0,pec=0,epc=0]@2023-07-26T15:10:10.2364773+03:00 - STARTED
|  |  |  |  +- SelectorProducer@58e1d9d
|  |  |  |  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59899ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  |  |  +> updates @ 2023-07-26T15:10:10.2364773+03:00 size=0
|  |  |  +> keys @ 2023-07-26T15:10:10.2364773+03:00 size=0
|  |  += oeji.ManagedSelector@446a1e84{STARTED} id=6 keys=0 selected=0 updates=0 - STARTED
|  |  |  += AdaptiveExecutionStrategy@4f0f2942/SelectorProducer@2657d4dd/PRODUCING/p=0/QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59898ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}][pc=0,pic=0,pec=0,epc=0]@2023-07-26T15:10:10.23747+03:00 - STARTED
|  |  |  |  +- SelectorProducer@2657d4dd
|  |  |  |  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59898ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  |  |  +> updates @ 2023-07-26T15:10:10.23747+03:00 size=0
|  |  |  +> keys @ 2023-07-26T15:10:10.23747+03:00 size=0
|  |  += oeji.ManagedSelector@5340477f{STARTED} id=7 keys=0 selected=0 updates=0 - STARTED
|  |     += AdaptiveExecutionStrategy@47caedad/SelectorProducer@7139992f/PRODUCING/p=0/QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59897ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}][pc=0,pic=0,pec=0,epc=0]@2023-07-26T15:10:10.2384667+03:00 - STARTED
|  |     |  +- SelectorProducer@7139992f
|  |     |  +~ QueuedThreadPool[qtp1415157681]@545997b1{STARTED,10<=10<=200,i=1,r=-1,t=59897ms,q=0}[ReservedThreadExecutor@673fdbce{reserved=0/16,pending=0}] - STARTED
|  |     +> updates @ 2023-07-26T15:10:10.2384667+03:00 size=0
|  |     +> keys @ 2023-07-26T15:10:10.2384667+03:00 size=0
|  +- sun.nio.ch.ServerSocketChannelImpl[/[0:0:0:0:0:0:0:0]:8080]
|  +- qtp1415157681-28-acceptor-0@3126dc4e-ServerConnector@183ec003{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
+- org.eclipse.jetty.server.Server$DynamicErrorHandler@69504ae9
+> startJarLoader@2e5c649
|  +> URLs size=14
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/base9/resources/
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/logging/slf4j-api-2.0.7.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/logging/jetty-slf4j-impl-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-client-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-alpn-client-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-http-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-server-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-xml-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-util-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-io-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-deploy-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-session-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-security-12.0.0-SNAPSHOT.jar
|  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-jndi-12.0.0-SNAPSHOT.jar
|  +> parent: jdk.internal.loader.ClassLoaders$AppClassLoader@2a139a55
|     +> packages size=4
|     |  +> package org.eclipse.jetty.start.config
|     |  +> package org.eclipse.jetty.start.builders
|     |  +> package org.eclipse.jetty.start.shaded.util
|     |  +> package org.eclipse.jetty.start
|     +> parent: jdk.internal.loader.ClassLoaders$PlatformClassLoader@5c7bfdc1
|        +> packages size=11
|           +> package sun.security.ec
|           +> package sun.security.pkcs11.wrapper
|           +> package sun.security.jgss
|           +> package com.sun.security.sasl.gsskerb
|           +> package sun.text.resources.cldr.ext
|           +> package sun.util.resources.provider
|           +> package org.jcp.xml.dsig.internal.dom
|           +> package sun.util.resources.cldr.provider
|           +> package sun.security.mscapi
|           +> package sun.security.smartcardio
|           +> package sun.security.pkcs11
+> environments size=2
|  +> oejuc.Environment$Named@0{core}
|  |  +> startJarLoader@2e5c649
|  |  |  +> URLs size=14
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/base9/resources/
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/logging/slf4j-api-2.0.7.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/logging/jetty-slf4j-impl-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-client-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-alpn-client-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-http-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-server-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-xml-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-util-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-io-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-deploy-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-session-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-security-12.0.0-SNAPSHOT.jar
|  |  |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-jndi-12.0.0-SNAPSHOT.jar
|  |  |  +> parent: jdk.internal.loader.ClassLoaders$AppClassLoader@2a139a55
|  |  |     +> packages size=4
|  |  |     |  +> package org.eclipse.jetty.start.config
|  |  |     |  +> package org.eclipse.jetty.start.builders
|  |  |     |  +> package org.eclipse.jetty.start.shaded.util
|  |  |     |  +> package org.eclipse.jetty.start
|  |  |     +> parent: jdk.internal.loader.ClassLoaders$PlatformClassLoader@5c7bfdc1
|  |  |        +> packages size=11
|  |  |           +> package sun.security.ec
|  |  |           +> package sun.security.pkcs11.wrapper
|  |  |           +> package sun.security.jgss
|  |  |           +> package com.sun.security.sasl.gsskerb
|  |  |           +> package sun.text.resources.cldr.ext
|  |  |           +> package sun.util.resources.provider
|  |  |           +> package org.jcp.xml.dsig.internal.dom
|  |  |           +> package sun.util.resources.cldr.provider
|  |  |           +> package sun.security.mscapi
|  |  |           +> package sun.security.smartcardio
|  |  |           +> package sun.security.pkcs11
|  |  +> Attributes core size=0
|  +> oejuc.Environment$Named@a0216c0{ee9}
|     +> java.net.URLClassLoader@38d8f54a
|     |  +> URLs size=30
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-jakarta-servlet-api-5.0.2.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-nested-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-servlet-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-security-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-webapp-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-plus-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.transaction-api-2.0.1.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.interceptor-api-2.0.1.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.enterprise.cdi-api-3.0.1.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jakarta.inject-api-2.0.1.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-annotations-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/asm-9.5.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/asm-analysis-9.5.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/asm-commons-9.5.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/asm-tree-9.5.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-annotations/jakarta.annotation-api-2.0.0.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-apache-jsp/org.eclipse.jdt.ecj-3.27.0.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-apache-jsp/org.mortbay.jasper.apache-el-10.0.14.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-apache-jsp/org.mortbay.jasper.apache-jsp-10.0.14.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-ee9-apache-jsp-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-glassfish-jstl/jakarta.servlet.jsp.jstl.jakarta.servlet.jsp.jstl-api-2.0.0.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-glassfish-jstl/org.glassfish.web.jakarta.servlet.jsp.jstl-2.0.0.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-websocket-core-common-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-websocket-core-client-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/jetty-websocket-core-server-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-servlet-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-jakarta-websocket-api-2.0.0.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jakarta-client-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jakarta-common-12.0.0-SNAPSHOT.jar
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/lib/ee9-websocket/jetty-ee9-websocket-jakarta-server-12.0.0-SNAPSHOT.jar
|     |  +> parent(core): startJarLoader@2e5c649
|     +> Attributes ee9 size=2
|        +> contextHandlerClass=org.eclipse.jetty.ee9.webapp.WebAppContext
|        +> org.eclipse.jetty.ee9.webapp.serverClasses=[Ljava.lang.String;@5276d6ee
+> attributes size=0
+> org.eclipse.jetty.util.resource.FileSystemPool@57bc27f5
   +> buckets size=0

How do I disable the server level CDI module? As I have not enabled it in my knowledge? The Enabled Modules I mentioned before does not mention CDI anywhere..

@andresluuk
Copy link
Author

I also tryed to get a dump with application deployed, but this one just overflows :P a bug?
andres.txt

@andresluuk
Copy link
Author

But yes If I kill the process before the overflow i see:

|     |  +> Systemclasses exp-weld40-beanClassAdd@1672fe87 size=15
|     |  |  +> jakarta.
|     |  |  +> java.
|     |  |  +> javax.
|     |  |  +> org.eclipse.jetty.ee9.jsp.
|     |  |  +> org.eclipse.jetty.ee9.servlet.DefaultServlet
|     |  |  +> org.eclipse.jetty.ee9.servlet.NoJspServlet

Maybe the default should not include jakarta.enterprise.* ? or maybe the jakarta.enterprise. is a undeclared depenecy of a module enabled for some reason..

@joakime
Copy link
Contributor

joakime commented Jul 26, 2023 •
edited

Do you see something like the following?

java.lang.IllegalStateException: Unable to locate CDIProvider
	at jakarta.enterprise.inject.spi.CDI.findAllProviders(CDI.java:139)
	at jakarta.enterprise.inject.spi.CDI.getCDIProvider(CDI.java:92)
	at jakarta.enterprise.inject.spi.CDI.current(CDI.java:65)
	at org.jboss.weld.module.web.servlet.WeldInitialListener.contextInitialized(WeldInitialListener.java:88)
	at org.jboss.weld.servlet.api.helpers.ForwardingServletListener.contextInitialized(ForwardingServletListener.java:34)
	at org.jboss.weld.environment.servlet.EnhancedListener.onStartup(EnhancedListener.java:66)
	at org.eclipse.jetty.ee10.servlet.ServletContainerInitializerHolder.doStart(ServletContainerInitializerHolder.java:153)

Notes for others researching this, the simplest way to replicate the above stacktrace is ...

$ mkdir cdi-base
$ cd cdi-base
$ java -jar /opt/jetty-home-12.0.0/start.jar --add-modules=http,ee10-deploy,ee10-annotations
$ cp ~/.m2/repository/org/eclipse/jetty/ee10/jetty-ee10-test-weld-cdi-webapp/12.0.0-SNAPSHOT/jetty-ee10-test-weld-cdi-webapp-12.0.0-SNAPSHOT.war webapps/demo-cdi-ee10.war
$ java -jar /opt/jetty-home-12.0.0/start.jar

@andresluuk
Copy link
Author

Yes, I started my investigation from there. and the jakarta.enterprise.inject.spi.CDI comes from the server and not my webapp. I have no idea why the server provides it, as I have not enabled servers CDI modules to my knowledge.

@joakime
Copy link
Contributor

joakime commented Jul 26, 2023

We have a replication case, no server level CDI enabled.
Lets see what's going on.

Oddly we have passing CDITests in our distribution tests.

@andresluuk
Copy link
Author

oh nice:)

java -jar /opt/jetty-home-12.0.0/start.jar --add-modules=http,ee10-deploy,ee10-annotations
From the modules you mentioned I have all of them enabled, but the list-modules does not declare it will include CDI as a dependency.
Mhh does ee10-annotations imply ee10-annotations?

@gregw
Copy link
Contributor

gregw commented Jul 26, 2023 •
edited

@joakime I vaguely recall that one of the updated jakarta API jars has a hard JPMS dependency on the CDI API. I think we are providing it by default because of that. I wonder if we can do some config magic so that it is hidden from the webapps unless we explicitly enable that as a module?

I'll investigate more soon

@gregw
Copy link
Contributor

gregw commented Jul 26, 2023

So we now have lib/jakarta.enterprise.cdi-api-3.0.1.jar as a dependency of ee9-plus and ee10-plus modules (which is needed for the eeX-annotations modules, which in turn are needed by websocket APIs and/or JSPs).
We don't actually use CDI at all, nor use anything that uses CDI, but we have to included it because JPMS needs it.

@sbordet I know this is REALLY late in the day.... but could we have dependencies that are only put on the classpath if we are running in jpms mode?

@gregw
Copy link
Contributor

gregw commented Jul 26, 2023

@andresluuk can you try running with an extra module enabled? Create the file modules/nocdi.mod in which you put:

[environment]
ee10

[ini]
jetty.webapp.addServerClasses+=,jakarta.enterprise.,jakarta.decorator.

then enable that module and run again. That should hide the CDI jar and might help.

@andresluuk
Copy link
Author

Thank you, I tryed it out and I got my CDI application working with it.

@andresluuk
Copy link
Author

Just a side note. I think the classloader dumping (jetty.server.dumpAfterStart=true) endless loop deserves also a check:

|     +> WebAppClassLoader{exp-weld40-beanClassAdd}@2235eaab
|     |  +> URLs size=22
|     |  |  +> file:/C:/work/servers/jetty-home-12.0.0-SNAPSHOT/webapps/exp-weld40-beanClassAdd/WEB-INF/classes/
...
|     +> handler attributes size=7
|     |  +> jakarta.servlet.context.tempdir: C:\Users\ALuuk\AppData\Local\Temp\jetty-0_0_0_0-8080-exp-weld40-beanClassAdd-_exp-weld40-beanClassAdd-any-8540187313042170303
|     |  +> org.eclipse.jetty.ee9.webapp.webFragments: ju.HashMap@0{size=0}
|     |  +> org.eclipse.jetty.jndi.EnvConfiguration: ju.ArrayList@1(size=0)
|     |  +> org.eclipse.jetty.resources: ju.HashSet@e852536b(size=1)
|     |  |     +: jar:file:///C:/work/servers/jetty-home-12.0.0-SNAPSHOT/webapps/exp-weld40-beanClassAdd/WEB-INF/lib/jakarta.faces-3.0.0.jar!/META-INF/resources/
|     |  |        +: jar:file:///C:/work/servers/jetty-home-12.0.0-SNAPSHOT/webapps/exp-weld40-beanClassAdd/WEB-INF/lib/jakarta.faces-3.0.0.jar!/META-INF/resources/
|     |  |           +: jar:file:///C:/work/servers/jetty-home-12.0.0-SNAPSHOT/webapps/exp-weld40-beanClassAdd/WEB-INF/lib/jakarta.faces-3.0.0.jar!/META-INF/resources/

The jakarta.faces-3.0.0 in the webapp is the default one from maven.

@sbordet
Copy link
Contributor

sbordet commented Jul 29, 2023 •
edited

@gregw wrote:

We don't actually use CDI at all, nor use anything that uses CDI, but we have to included it because JPMS needs it.

I'm not sure I understand the problem?
Did you find what jakarta jar lists CDI as a non-static dependency in its module-info?

If there is no hard dependency on the module-info, then we also should not have CDI as a hard dependency and all will work.

If there is a jakarta jar that lists CDI as a non-static dependency, but that is not required, then it's a bug in the jakarta jar.
Rather than working it around, I would ask them to fix it, and meanwhile we only run in non-JPMS mode.

@gregw
Copy link
Contributor

gregw commented Jul 31, 2023

I don't know if they don't use CDI or if it is just that we don't use any API that needs CDI. So I know is that we never needed the CDI dependency before, but now we do because of JPMS. I'll drive deeper the next few days.

@gregw
Copy link
Contributor

gregw commented Jul 31, 2023

@sbordet The jakarta transaction api does use some classes from the jakarta cdi api, specifically:

  • jakarta.enterprise.util.Nonbinding annotation
  • jakarta.enterprice.context.NormalScope annotation

So it looks like they are real dependencies even though they are not in the main API, they are annotations that CDI will look for.

Thus we now have a hard dependency on a specific version of the CDI API which we are currently exposing to the webapp. Thus we need to need to hide these APIs from webapps, that might have different versions of CDI.
But then if we provide CDI, we will have to unhide.

@andresluuk
Copy link
Author

What is the final result here on jetty 12.0.0 when it is released. Should I keep the special custom module, or it is no longer needed?

@gregw
Copy link
Contributor

gregw commented Aug 3, 2023

For now the custom module will still be required. We need to gain more experience with his the change in Jakarta APIs to depend on CDI affects deployments.

I.e. is there a bug (so we should hide it) out a feature (we should embrace it and maybe even provide a CDI implementation).

So for now if you use any jetty plus features (annotations, JSP, jndi) then you will get a CDI API in your class loader. If the webapp wants to use CDI, then either it needs to be at the same version or do a custom module to hide the container provided version.

@andresluuk
Copy link
Author

Ok, I will continue with the special module then.

If the webapp wants to use CDI, then either it needs to be at the same version ..

Just to clarify by "same version" you mean here use the CDI provided by the jetty, as the issue was that because of the API on the server path you can't provide any (including one matching the servers CDI apis) CDI versions by the webapp classpath.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@gregw @joakime @sbordet @andresluuk