Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

play framework 1.4 hang during sending requests to oracle database #1044

Closed
karolszk opened this issue Dec 17, 2016 · 4 comments
Closed

play framework 1.4 hang during sending requests to oracle database #1044

karolszk opened this issue Dec 17, 2016 · 4 comments

Comments

@karolszk
Copy link

Hello,

I have an application which causes a lot of requests to work for oracle and hangs. I don't have any idea what was reason for such behavior. (When I'm not using the app automatically is unblocked after some period of time (15-20minutes) which is unacceptable to production use. Normally application is used by 50 users.

The problem is repeatable (there are easy steps to follow by one user and go into this hang) and changing from 1.4.2 to 1.4.3 do nothing.
And adding -Dorg.jboss.netty.epollBugWorkaround=true also doesn't help.

and here jstack:

"New I/O boss #27" #63 prio=5 os_prio=0 tid=0x0000000035a22000 nid=0xe70 runnable [0x000000003cd0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x000000052762cc58> (a sun.nio.ch.Util$2)
- locked <0x000000052762cc48> (a java.util.Collections$UnmodifiableSet)
- locked <0x000000052762c9d8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x000000052762c6b8> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #26" #61 daemon prio=5 os_prio=0 tid=0x0000000034d52000 nid=0xaa8 runnable [0x000000003cc0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005276094a8> (a sun.nio.ch.Util$2)
- locked <0x0000000527609498> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000527609348> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000527609020> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #25" #60 daemon prio=5 os_prio=0 tid=0x0000000034d53800 nid=0xd78 runnable [0x000000003cb0f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000527621300> (a sun.nio.ch.Util$2)
- locked <0x00000005276212f0> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005276211a0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000527620e78> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #24" #59 daemon prio=5 os_prio=0 tid=0x0000000034d51800 nid=0xbb0 runnable [0x000000003ca0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000527616018> (a sun.nio.ch.Util$2)
- locked <0x0000000527616008> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000527615eb8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000527615b90> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #23" #58 daemon prio=5 os_prio=0 tid=0x0000000034d50800 nid=0x3e0 runnable [0x000000003c90f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000527614560> (a sun.nio.ch.Util$2)
- locked <0x0000000527614550> (a java.util.Collections$UnmodifiableSet)
- locked <0x000000052760f388> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x000000052760f060> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #22" #57 daemon prio=5 os_prio=0 tid=0x0000000034d53000 nid=0xbb4 runnable [0x000000003c80e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000527609f48> (a sun.nio.ch.Util$2)
- locked <0x0000000527609f38> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000527609de8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000527609ac0> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #21" #56 daemon prio=5 os_prio=0 tid=0x0000000034d4d000 nid=0x13fc runnable [0x000000003c70f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000527619b00> (a sun.nio.ch.Util$2)
- locked <0x0000000527619af0> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005276199a0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000527619678> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #20" #55 daemon prio=5 os_prio=0 tid=0x0000000034d4e800 nid=0xfcc runnable [0x000000003c60e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x000000052760ea48> (a sun.nio.ch.Util$2)
- locked <0x000000052760ea38> (a java.util.Collections$UnmodifiableSet)
- locked <0x000000052760e8e8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x000000052760e5c0> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #19" #54 daemon prio=5 os_prio=0 tid=0x0000000034d4d800 nid=0x1218 runnable [0x000000003c50e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x000000052761f7f8> (a sun.nio.ch.Util$2)
- locked <0x000000052761f7e8> (a java.util.Collections$UnmodifiableSet)
- locked <0x000000052761f588> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x000000052761a118> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"Hashed wheel timer #1" #53 prio=5 os_prio=0 tid=0x0000000034d50000 nid=0x718 waiting on condition [0x000000003c40e000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.jboss.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:445)
at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:364)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- None

"play-thread-1" #52 prio=5 os_prio=0 tid=0x0000000034d4f000 nid=0x4a8 runnable [0x000000003c10d000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at oracle.net.ns.Packet.receive(Packet.java:300)
at oracle.net.ns.DataPacket.receive(DataPacket.java:106)
at oracle.net.ano.CryptoDataPacket.receive(Unknown Source)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:260)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:185)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:102)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:884)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1167)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1289)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3628)
- locked <0x000000053e04b6c0> (a oracle.jdbc.driver.T4CConnection)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1493)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:1418)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:79)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2062)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1859)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:906)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doList(Loader.java:2550)
at org.hibernate.loader.Loader.doList(Loader.java:2536)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2366)
at org.hibernate.loader.Loader.list(Loader.java:2361)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:495)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:357)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:198)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
at play.db.jpa.GenericModel$JPAQuery.fetch(GenericModel.java:449)
at ext.Paging.fetch(Paging.java:86)
at controllers.utilities.QueryByCriteriaSwitchOffFactory.getListAndSetCountPage(QueryByCriteriaSwitchOffFactory.java:202)
at services.zzzzzz.OrderService.getListOrdersByCriteriaSwitchOff(OrderService.java:148)
at actions.reports.SwitchOffAction.execute(SwitchOffAction.java:40)
at controllers.Reports.switchOff(Reports.java:357)
at sun.reflect.GeneratedMethodAccessor1908.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:544)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:494)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:489)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:458)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:162)
at play.server.PlayHandler$NettyInvocation.execute(PlayHandler.java:326)
at play.Invoker$Invocation$1.apply(Invoker.java:292)
at play.Invoker$Invocation$1.apply(Invoker.java:288)
at play.db.jpa.JPA.withTransaction(JPA.java:258)
at play.db.jpa.JPA.withinFilter(JPA.java:217)
at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
at play.Invoker$Invocation.withinFilter(Invoker.java:271)
at play.Invoker$Invocation.run(Invoker.java:288)
at play.server.PlayHandler$NettyInvocation.run(PlayHandler.java:303)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005274da428> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"DestroyJavaVM" #51 prio=5 os_prio=0 tid=0x0000000034d49000 nid=0xab4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:
- None

"New I/O server boss #18" #50 prio=5 os_prio=0 tid=0x0000000034d4c000 nid=0x1088 runnable [0x000000003be0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275f8360> (a sun.nio.ch.Util$2)
- locked <0x00000005275f8350> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275f7e88> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.jboss.netty.channel.socket.nio.NioServerBoss.select(NioServerBoss.java:163)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275f7b60> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #17" #49 prio=5 os_prio=0 tid=0x0000000034d4b800 nid=0x1308 runnable [0x000000003bd0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275e8890> (a sun.nio.ch.Util$2)
- locked <0x00000005275e8880> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275e36b8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275e3398> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #16" #48 prio=5 os_prio=0 tid=0x0000000034d48800 nid=0x1364 runnable [0x000000003bc0f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275f3930> (a sun.nio.ch.Util$2)
- locked <0x00000005275f3920> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275ee758> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275ee438> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #15" #47 prio=5 os_prio=0 tid=0x0000000034d4a800 nid=0xdec runnable [0x000000003bb0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275dd7f0> (a sun.nio.ch.Util$2)
- locked <0x00000005275dd7e0> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275d86b8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275d8398> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #14" #46 prio=5 os_prio=0 tid=0x0000000034d47800 nid=0xe8c runnable [0x000000003ba0f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275e2d80> (a sun.nio.ch.Util$2)
- locked <0x00000005275e2d70> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275e2c20> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275e2900> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #13" #45 prio=5 os_prio=0 tid=0x0000000034d47000 nid=0x1284 runnable [0x000000003b90e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275ede20> (a sun.nio.ch.Util$2)
- locked <0x00000005275ede10> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275edcc0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275ed9a0> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #12" #44 prio=5 os_prio=0 tid=0x0000000034d4a000 nid=0x1294 runnable [0x000000003b80f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275ea340> (a sun.nio.ch.Util$2)
- locked <0x00000005275ea330> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275ea1e0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275e9ec0> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #11" #43 prio=5 os_prio=0 tid=0x0000000034d45800 nid=0x1068 runnable [0x000000003b70e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275df2a0> (a sun.nio.ch.Util$2)
- locked <0x00000005275df290> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275df140> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275dee20> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #10" #42 prio=5 os_prio=0 tid=0x0000000034d44800 nid=0x1164 runnable [0x000000003b60e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000005275d7d80> (a sun.nio.ch.Util$2)
- locked <0x00000005275d7d70> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000005275d7c20> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275d7900> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O server boss #9" #41 prio=5 os_prio=0 tid=0x0000000034d46000 nid=0xe00 runnable [0x000000003b50e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525a6e738> (a sun.nio.ch.Util$2)
- locked <0x0000000525a6e728> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525a6e1f0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.jboss.netty.channel.socket.nio.NioServerBoss.select(NioServerBoss.java:163)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525a6dc60> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #8" #40 prio=5 os_prio=0 tid=0x000000003394d000 nid=0xe94 runnable [0x000000003b40f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525a8efd0> (a sun.nio.ch.Util$2)
- locked <0x0000000525a8efc0> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525a8ee60> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525a8ebd8> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #7" #39 prio=5 os_prio=0 tid=0x0000000030a53000 nid=0x944 runnable [0x000000003b30f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525a7b960> (a sun.nio.ch.Util$2)
- locked <0x0000000525a7b950> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525a7b7f0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525a7b430> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #6" #38 prio=5 os_prio=0 tid=0x0000000030a4f800 nid=0x670 runnable [0x000000003b20f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525adcb38> (a sun.nio.ch.Util$2)
- locked <0x0000000525adcb28> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525adc918> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525adc600> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #5" #37 prio=5 os_prio=0 tid=0x0000000030a52800 nid=0xf50 runnable [0x000000003b10e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525a6ede0> (a sun.nio.ch.Util$2)
- locked <0x0000000525a6edd0> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525a6d850> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525a6d500> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #4" #36 prio=5 os_prio=0 tid=0x0000000030a51800 nid=0x126c runnable [0x000000003b00e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525a8d408> (a sun.nio.ch.Util$2)
- locked <0x0000000525a8d3f8> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525a8d1e8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525a88ed0> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #3" #35 prio=5 os_prio=0 tid=0x0000000030a4e800 nid=0x7b0 runnable [0x000000003af0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525a87700> (a sun.nio.ch.Util$2)
- locked <0x0000000525a876f0> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525a874e0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525a871c8> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #2" #34 prio=5 os_prio=0 tid=0x0000000030a51000 nid=0xbf8 runnable [0x000000003ae0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525a75bc0> (a sun.nio.ch.Util$2)
- locked <0x0000000525a75bb0> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525a759a0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525a75648> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"New I/O worker #1" #33 prio=5 os_prio=0 tid=0x0000000030a50000 nid=0xf1c runnable [0x000000003ad0e000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x0000000525b62890> (a sun.nio.ch.Util$2)
- locked <0x0000000525b62880> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000525b62670> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x0000000525b62358> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"jobs-thread-10" #32 prio=5 os_prio=0 tid=0x0000000030a4b000 nid=0x1098 waiting on condition [0x000000003ac0e000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000005b8060540> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at play.libs.F$Promise.get(F.java:60)
at services.plannedWorks.registry.queue.DateRangeUpdate.update(DateRangeUpdate.java:18)
at services.plannedWorks.registry.queue.UpdateQueue.tryUpdateDateRange(UpdateQueue.java:77)
at services.plannedWorks.registry.queue.UpdateQueue.updateAll(UpdateQueue.java:56)
- locked <0x00000005275a88e0> (a services.plannedWorks.registry.queue.UpdateQueue)
at services.plannedWorks.registry.queue.UpdateQueueJob.doJob(UpdateQueueJob.java:18)
at play.jobs.Job.doJobWithResult(Job.java:54)
at play.jobs.Job$2.apply(Job.java:208)
at play.db.jpa.JPA.withTransaction(JPA.java:258)
at play.db.jpa.JPA.withinFilter(JPA.java:217)
at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
at play.jobs.Job.withinFilter(Job.java:185)
at play.jobs.Job.call(Job.java:204)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Regards, Karol

@asolntsev
Copy link
Contributor

Hi @karolszk

  1. You have a very unoptimized query from database: controllers.utilities.QueryByCriteriaSwitchOffFactory.getListAndSetCountPage(QueryByCriteriaSwitchOffFactory.java:202. You need to optimize it. Probably you should add an index to database. And probably cache the result.

  2. You have only one worker thread in Play pool: play-thread-1. Then this thread is busy with the slow DB query, Play cannot handle other requests. Try to increase play pool size by adding the following line in application.conf:
    play.pool=10

@karolszk
Copy link
Author

Hi @asolntsev thanks for help :

ad. 1 - OK. I try to optimize it.
ad. 2. I had play.pool=8

please look at the rest of the play-threads-* - I've added missing now:

"play-thread-8" #70 prio=5 os_prio=0 tid=0x0000000035a2a000 nid=0x6b4 waiting on condition [0x000000003d80e000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000005274ceb60> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- None

"play-thread-7" #69 prio=5 os_prio=0 tid=0x0000000035a29800 nid=0x9e0 runnable [0x000000003d70d000]
java.lang.Thread.State: RUNNABLE
at org.hibernate.event.internal.AbstractVisitor.processValue(AbstractVisitor.java:119)
at org.hibernate.event.internal.WrapVisitor.processValue(WrapVisitor.java:125)
at org.hibernate.event.internal.AbstractVisitor.processEntityPropertyValues(AbstractVisitor.java:76)
at org.hibernate.event.internal.DefaultFlushEntityEventListener.wrapCollections(DefaultFlushEntityEventListener.java:223)
at org.hibernate.event.internal.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:162)
at org.hibernate.event.internal.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:251)
at org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:100)
at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:61)
at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1166)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1223)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
at play.db.jpa.GenericModel$JPAQuery.fetch(GenericModel.java:406)
at models.Work.getEffectTags(Work.java:196)
at sun.reflect.GeneratedMethodAccessor398.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:196)
at models.Work_$$jvstb_d3.getEffectTags(Work$$_jvstb_d3.java)
at services.damages.DamageService.setShowOnMapDto(DamageService.java:262)
at services.damages.DamageService.addShowOnMapDetails(DamageService.java:249)
at models.damages.Damage.getDamages(Damage.java:270)
at models.damages.Damage.getAllDamages(Damage.java:244)
at models.utilities.ExportData.DamagesToFrReport.(DamagesToFrReport.java:95)
at controllers.Damages.export(Damages.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:544)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:494)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:489)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:458)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:162)
at play.server.PlayHandler$NettyInvocation.execute(PlayHandler.java:326)
at play.Invoker$Invocation$1.apply(Invoker.java:292)
at play.Invoker$Invocation$1.apply(Invoker.java:288)
at play.db.jpa.JPA.withTransaction(JPA.java:258)
at play.db.jpa.JPA.withinFilter(JPA.java:217)
at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
at play.Invoker$Invocation.withinFilter(Invoker.java:271)
at play.Invoker$Invocation.run(Invoker.java:288)
at play.server.PlayHandler$NettyInvocation.run(PlayHandler.java:303)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005274e46a0> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"play-thread-6" #68 prio=5 os_prio=0 tid=0x0000000035a25000 nid=0x1120 runnable [0x000000003d60d000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at oracle.net.ns.Packet.receive(Packet.java:300)
at oracle.net.ns.DataPacket.receive(DataPacket.java:106)
at oracle.net.ano.CryptoDataPacket.receive(Unknown Source)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:260)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:185)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:102)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:884)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1167)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1289)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3628)
- locked <0x000000052721fdc8> (a oracle.jdbc.driver.T4CConnection)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1493)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:1418)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:79)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2062)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1859)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:906)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doList(Loader.java:2550)
at org.hibernate.loader.Loader.doList(Loader.java:2536)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2366)
at org.hibernate.loader.Loader.list(Loader.java:2361)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:495)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:357)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:198)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
at play.db.jpa.GenericModel$JPAQuery.fetch(GenericModel.java:406)
at models.Work.getEffectTags(Work.java:196)
at services.exportData.outage.RecordEventExportOutage.setDataReportFromWorkAttributes(RecordEventExportOutage.java:254)
at services.exportData.outage.RecordEventExportOutage.(RecordEventExportOutage.java:136)
at models.utilities.query.QueryDisconnectedLineOutage$3.apply(QueryDisconnectedLineOutage.java:532)
at models.utilities.query.QueryDisconnectedLineOutage$3.apply(QueryDisconnectedLineOutage.java:1)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:301)
at com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.java:691)
at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:275)
at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:226)
at com.google.common.collect.FluentIterable.toList(FluentIterable.java:334)
at models.utilities.query.QueryDisconnectedLineOutage.conversionListExportEvent(QueryDisconnectedLineOutage.java:534)
at models.utilities.query.QueryDisconnectedLineOutage.getResultListExport(QueryDisconnectedLineOutage.java:464)
at models.zzzzzz.DisconnectedLine.getDisconnectedObjectExport(DisconnectedLine.java:174)
at services.zzzzzz.disconnectedLine.DisconnectedLineService.getListToExportData(DisconnectedLineService.java:109)
at services.exportData.outage.ExportEventService.findData(ExportEventService.java:41)
at services.exportData.outage.ExportEventService.export(ExportEventService.java:31)
at actions.outage.exportData.ExportEventAction.execute(ExportEventAction.java:30)
at controllers.Reports.exportOutageEvent(Reports.java:589)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:544)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:494)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:489)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:458)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:162)
at play.server.PlayHandler$NettyInvocation.execute(PlayHandler.java:326)
at play.Invoker$Invocation$1.apply(Invoker.java:292)
at play.Invoker$Invocation$1.apply(Invoker.java:288)
at play.db.jpa.JPA.withTransaction(JPA.java:258)
at play.db.jpa.JPA.withinFilter(JPA.java:217)
at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
at play.Invoker$Invocation.withinFilter(Invoker.java:271)
at play.Invoker$Invocation.run(Invoker.java:288)
at play.server.PlayHandler$NettyInvocation.run(PlayHandler.java:303)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005274cfc60> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"play-thread-5" #67 prio=5 os_prio=0 tid=0x0000000035a28000 nid=0xc1c runnable [0x000000003d50d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.intern(Native Method)
at java.lang.Class.searchMethods(Class.java:3003)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at play.classloading.enhancers.PropertiesEnhancer$FieldAccessor.invokeReadProperty(PropertiesEnhancer.java:223)
at services.zzzzzz.disconnectedLine.DisconnectedLineService.loadDisconnectedLine(DisconnectedLineService.java:129)
at models.utilities.ExportData.outage.ReportsMergedOutageSimpleToFrReport.setTableBody(ReportsMergedOutageSimpleToFrReport.java:98)
at models.utilities.ExportData.outage.ReportsMergedOutageSimpleToFrReport.(ReportsMergedOutageSimpleToFrReport.java:55)
at controllers.Reports.exportOutageSimple(Reports.java:622)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:544)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:494)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:489)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:458)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:162)
at play.server.PlayHandler$NettyInvocation.execute(PlayHandler.java:326)
at play.Invoker$Invocation$1.apply(Invoker.java:292)
at play.Invoker$Invocation$1.apply(Invoker.java:288)
at play.db.jpa.JPA.withTransaction(JPA.java:258)
at play.db.jpa.JPA.withinFilter(JPA.java:217)
at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
at play.Invoker$Invocation.withinFilter(Invoker.java:271)
at play.Invoker$Invocation.run(Invoker.java:288)
at play.server.PlayHandler$NettyInvocation.run(PlayHandler.java:303)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005274d88d0> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"play-thread-4" #66 prio=5 os_prio=0 tid=0x0000000035a24000 nid=0x3d8 runnable [0x000000003d40c000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at oracle.net.ns.Packet.receive(Packet.java:300)
at oracle.net.ns.DataPacket.receive(DataPacket.java:106)
at oracle.net.ano.CryptoDataPacket.receive(Unknown Source)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:260)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:185)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:102)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:884)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1167)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1289)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3628)
- locked <0x00000005c3d45188> (a oracle.jdbc.driver.T4CConnection)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1493)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:1418)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:79)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2062)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1859)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:906)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:318)
at org.hibernate.loader.Loader.loadEntity(Loader.java:2145)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:82)
at org.hibernate.loader.entity.EntityLoader.loadByUniqueKey(EntityLoader.java:161)
at org.hibernate.persister.entity.AbstractEntityPersister.loadByUniqueKey(AbstractEntityPersister.java:2374)
at org.hibernate.type.EntityType.loadByUniqueKey(EntityType.java:765)
at org.hibernate.type.EntityType.resolve(EntityType.java:505)
at org.hibernate.engine.internal.TwoPhaseLoad.doInitializeEntity(TwoPhaseLoad.java:168)
at org.hibernate.engine.internal.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:137)
at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:1112)
at org.hibernate.loader.Loader.processResultSet(Loader.java:969)
at org.hibernate.loader.Loader.doQuery(Loader.java:917)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:318)
at org.hibernate.loader.Loader.loadEntity(Loader.java:2145)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:82)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:72)
at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:3939)
at org.hibernate.event.internal.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:462)
at org.hibernate.event.internal.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:431)
at org.hibernate.event.internal.DefaultLoadEventListener.load(DefaultLoadEventListener.java:208)
at org.hibernate.event.internal.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:264)
at org.hibernate.event.internal.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:152)
at org.hibernate.internal.SessionImpl.fireLoad(SessionImpl.java:1053)
at org.hibernate.internal.SessionImpl.internalLoad(SessionImpl.java:980)
at org.hibernate.type.EntityType.resolveIdentifier(EntityType.java:714)
at org.hibernate.type.EntityType.resolve(EntityType.java:502)
at org.hibernate.engine.internal.TwoPhaseLoad.doInitializeEntity(TwoPhaseLoad.java:168)
at org.hibernate.engine.internal.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:137)
at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:1112)
at org.hibernate.loader.Loader.processResultSet(Loader.java:969)
at org.hibernate.loader.Loader.doQuery(Loader.java:917)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doList(Loader.java:2550)
at org.hibernate.loader.Loader.doList(Loader.java:2536)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2366)
at org.hibernate.loader.Loader.list(Loader.java:2361)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:495)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:357)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:198)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
at play.db.jpa.GenericModel$JPAQuery.fetch(GenericModel.java:406)
at ext.Paging.fetch(Paging.java:83)
at models.damages.Damage.getDamages(Damage.java:260)
at models.damages.Damage.getAllDamages(Damage.java:244)
at models.utilities.ExportData.DamagesToFrReport.(DamagesToFrReport.java:95)
at controllers.Damages.export(Damages.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:544)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:494)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:489)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:458)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:162)
at play.server.PlayHandler$NettyInvocation.execute(PlayHandler.java:326)
at play.Invoker$Invocation$1.apply(Invoker.java:292)
at play.Invoker$Invocation$1.apply(Invoker.java:288)
at play.db.jpa.JPA.withTransaction(JPA.java:258)
at play.db.jpa.JPA.withinFilter(JPA.java:217)
at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
at play.Invoker$Invocation.withinFilter(Invoker.java:271)
at play.Invoker$Invocation.run(Invoker.java:288)
at play.server.PlayHandler$NettyInvocation.run(PlayHandler.java:303)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005274e2e68> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"play-thread-3" #65 prio=5 os_prio=0 tid=0x0000000035a27000 nid=0x940 runnable [0x000000003d30d000]
java.lang.Thread.State: RUNNABLE
at org.hibernate.event.internal.AbstractVisitor.processValue(AbstractVisitor.java:123)
at org.hibernate.event.internal.WrapVisitor.processValue(WrapVisitor.java:125)
at org.hibernate.event.internal.AbstractVisitor.processEntityPropertyValues(AbstractVisitor.java:76)
at org.hibernate.event.internal.DefaultFlushEntityEventListener.wrapCollections(DefaultFlushEntityEventListener.java:223)
at org.hibernate.event.internal.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:162)
at org.hibernate.event.internal.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:251)
at org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:100)
at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:61)
at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1166)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1223)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
at dao.netEvents.NetEventDao.findByDamageId(NetEventDao.java:102)
at services.netEvents.NetEventService.getByDamageId(NetEventService.java:124)
at models.damages.DamageBase.getLowVoltageCircuitByWork(DamageBase.java:251)
at models.damages.DamageBase.getSubstationAndObjectName(DamageBase.java:348)
at models.utilities.ExportData.DamagesToFrReport.getObjectName(DamagesToFrReport.java:372)
at models.utilities.ExportData.DamagesToFrReport.prepareReport(DamagesToFrReport.java:301)
at models.utilities.ExportData.DamagesToFrReport.(DamagesToFrReport.java:115)
at controllers.Damages.export(Damages.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:544)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:494)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:489)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:458)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:162)
at play.server.PlayHandler$NettyInvocation.execute(PlayHandler.java:326)
at play.Invoker$Invocation$1.apply(Invoker.java:292)
at play.Invoker$Invocation$1.apply(Invoker.java:288)
at play.db.jpa.JPA.withTransaction(JPA.java:258)
at play.db.jpa.JPA.withinFilter(JPA.java:217)
at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
at play.Invoker$Invocation.withinFilter(Invoker.java:271)
at play.Invoker$Invocation.run(Invoker.java:288)
at play.server.PlayHandler$NettyInvocation.run(PlayHandler.java:303)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005275c2b98> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"play-thread-2" #64 prio=5 os_prio=0 tid=0x0000000035a23800 nid=0x694 runnable [0x000000003d20d000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at oracle.net.ns.Packet.receive(Packet.java:300)
at oracle.net.ns.DataPacket.receive(DataPacket.java:106)
at oracle.net.ano.CryptoDataPacket.receive(Unknown Source)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:260)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:185)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:102)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:884)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1167)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1289)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3628)
- locked <0x00000004c08ce1c0> (a oracle.jdbc.driver.T4CConnection)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1493)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:1418)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:79)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2062)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1859)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:906)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doList(Loader.java:2550)
at org.hibernate.loader.Loader.doList(Loader.java:2536)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2366)
at org.hibernate.loader.Loader.list(Loader.java:2361)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:495)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:357)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:198)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
at play.db.jpa.GenericModel$JPAQuery.fetch(GenericModel.java:449)
at ext.Paging.fetch(Paging.java:86)
at controllers.utilities.QueryByCriteriaSwitchOffFactory.getListAndSetCountPage(QueryByCriteriaSwitchOffFactory.java:202)
at services.zzzzzz.OrderService.getListOrdersByCriteriaSwitchOff(OrderService.java:148)
at actions.reports.SwitchOffAction.execute(SwitchOffAction.java:40)
at controllers.Reports.switchOff(Reports.java:357)
at sun.reflect.GeneratedMethodAccessor1908.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:544)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:494)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:489)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:458)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:162)
at play.server.PlayHandler$NettyInvocation.execute(PlayHandler.java:326)
at play.Invoker$Invocation$1.apply(Invoker.java:292)
at play.Invoker$Invocation$1.apply(Invoker.java:288)
at play.db.jpa.JPA.withTransaction(JPA.java:258)
at play.db.jpa.JPA.withinFilter(JPA.java:217)
at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
at play.Invoker$Invocation.withinFilter(Invoker.java:271)
at play.Invoker$Invocation.run(Invoker.java:288)
at play.server.PlayHandler$NettyInvocation.run(PlayHandler.java:303)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Locked ownable synchronizers:
- <0x00000005274e3718> (a java.util.concurrent.ThreadPoolExecutor$Worker)

@karolszk
Copy link
Author

Hi again, @asolntsev "Try to increase play pool size by adding the following line in application.conf:
play.pool=10"
how big may be this value? From doc number_of_processors+1 but ts it possible to increase to 50? :)
Regards, Karol

@asolntsev
Copy link
Contributor

Hi @karolszk !
Most of your "play-thread-*" threads are busy with database operations. So, I repeat: you need to analyze your selects and probably add some indexes to database.

I think that recommendation "doc number_of_processors+1" is only valid for applications that use asynchronous operations everywhere (like Rest services calls). In other words, threads "play-thread-*" never wait for anything in such applications.

Oracle jdbc driver is not asynochronous, meaning that your threads do wait for database response. In this case I recommend setting `play.pool=database pool + N".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants