Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stabilizer Hz-3.2.3. cluster = "xLarge" client Node, OperationTimeoutException #2778

Closed
Danny-Hazelcast opened this issue Jun 20, 2014 · 1 comment
Closed

Stabilizer Hz-3.2.3. cluster = "xLarge" client Node, OperationTimeoutException #2778

Danny-Hazelcast opened this issue Jun 20, 2014 · 1 comment
Labels
Source: Internal PR or issue was opened by an employee Team: Performance

Comments

@Danny-Hazelcast
Copy link
Contributor

////////////////////////////
test.properties
StringMapTest.class=com.hazelcast.stabilizer.tests.map.StringMapTest
StringMapTest.threadCount=3
StringMapTest.keyLength=300
StringMapTest.valueLength=300
StringMapTest.keyCount=100000
StringMapTest.valueCount=100000
StringMapTest.writePercentage=20
StringMapTest.basename=StringMapTest

MapCasTest.class=com.hazelcast.stabilizer.tests.map.MapCasTest
MapCasTest.threadCount=3
MapCasTest.keyCount=1000
MapCasTest.basename=MapCasTest

MapLockTest.class=com.hazelcast.stabilizer.tests.map.MapLockTest
MapLockTest.threadCount=3
MapLockTest.keyCount=1000
MapLockTest.basename=com.MapLockTest

MapTransactionTest.class=com.hazelcast.stabilizer.tests.map.MapTransactionTest
MapTransactionTest.threadCount=3
MapTransactionTest.keyCount=1000
MapTransactionTest.basename=MapTransactionTest

QueueTest.class=com.hazelcast.stabilizer.tests.queue.QueueTest
QueueTest.queueLength=3
QueueTest.threadsPerQueue=1
QueueTest.messagesPerQueue=7
QueueTest.basename=QueueTest
////////////////////////////

ec2 stabilizer host box
/home/ec2-user/stabilizer-tests/final/xlarge/failures-1402675821962.txt

Failure[
message='Worked ran into an unhandled exception'
type='Worker exception'
agentAddress=10.101.159.173
time=Fri Jun 13 18:14:44 UTC 2014
workerAddress=client:10.101.159.173
workerId=worker-10.101.159.173-3-client
test=TestCase{
id=MapTransactionTest
, class=com.hazelcast.stabilizer.tests.map.MapTransactionTest
, basename=MapTransactionTest
, keyCount=1000
, threadCount=3
}
cause=com.hazelcast.core.OperationTimeoutException: No response for 120000 ms. Aborting invocation! InvocationFuture{invocation=BasicInvocation{ serviceName='hz:impl:mapService', op=TxnLockAndGetOperation{timeout=120000, thread=49}, partitionId=248, replicaIndex=0, tryCount=250, tryPauseMillis=500, invokeCount=1, callTimeout=60000, target=Address[10.237.187.71]:5701}, done=false} No response has been send backups-expected: 0 backups-completed: 0 reinvocations: 0
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.newOperationTimeoutException(BasicInvocation.java:782)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.waitForResponse(BasicInvocation.java:760)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:697)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676)
at com.hazelcast.map.tx.TransactionalMapProxySupport.lockAndGet(TransactionalMapProxySupport.java:196)
at com.hazelcast.map.tx.TransactionalMapProxySupport.getForUpdateInternal(TransactionalMapProxySupport.java:104)
at com.hazelcast.map.tx.TransactionalMapProxy.getForUpdate(TransactionalMapProxy.java:88)
at com.hazelcast.map.client.AbstractTxnMapRequest.innerCall(AbstractTxnMapRequest.java:96)
at com.hazelcast.client.txn.BaseTransactionRequest.call(BaseTransactionRequest.java:40)
at com.hazelcast.client.CallableClientRequest.process(CallableClientRequest.java:27)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.processRequest(ClientEngineImpl.java:499)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:446)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.util.executor.PoolExecutorThreadFactory$ManagedThread.run(PoolExecutorThreadFactory.java:59)
at ------ End remote and begin local stack-trace ------.(Unknown Source)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:862)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:795)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676)
at com.hazelcast.map.tx.TransactionalMapProxySupport.lockAndGet(TransactionalMapProxySupport.java:196)
at com.hazelcast.map.tx.TransactionalMapProxySupport.getForUpdateInternal(TransactionalMapProxySupport.java:104)
at com.hazelcast.map.tx.TransactionalMapProxy.getForUpdate(TransactionalMapProxy.java:88)
at com.hazelcast.map.client.AbstractTxnMapRequest.innerCall(AbstractTxnMapRequest.java:96)
at com.hazelcast.client.txn.BaseTransactionRequest.call(BaseTransactionRequest.java:40)
at com.hazelcast.client.CallableClientRequest.process(CallableClientRequest.java:27)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.processRequest(ClientEngineImpl.java:499)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:446)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.util.executor.PoolExecutorThreadFactory$ManagedThread.run(PoolExecutorThreadFactory.java:59)
at ------ End remote and begin local stack-trace ------.(Unknown Source)
at com.hazelcast.client.spi.impl.ClientCallFuture.resolveResponse(ClientCallFuture.java:160)
at com.hazelcast.client.spi.impl.ClientCallFuture.get(ClientCallFuture.java:109)
at com.hazelcast.client.spi.impl.ClientCallFuture.get(ClientCallFuture.java:90)
at com.hazelcast.client.txn.proxy.ClientTxnProxy.invoke(ClientTxnProxy.java:54)
at com.hazelcast.client.txn.proxy.ClientTxnMapProxy.getForUpdate(ClientTxnMapProxy.java:55)
at com.hazelcast.stabilizer.tests.map.MapTransactionTest$Worker$1.execute(MapTransactionTest.java:124)
at com.hazelcast.client.txn.ClientTransactionManager.executeTransaction(ClientTransactionManager.java:72)
at com.hazelcast.client.txn.ClientTransactionManager.executeTransaction(ClientTransactionManager.java:65)
at com.hazelcast.client.HazelcastClient.executeTransaction(HazelcastClient.java:299)
at com.hazelcast.client.HazelcastClientProxy.executeTransaction(HazelcastClientProxy.java:135)
at com.hazelcast.stabilizer.tests.map.MapTransactionTest$Worker.run(MapTransactionTest.java:120)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.stabilizer.tests.utils.ThreadSpawner$DefaultThread.run(ThreadSpawner.java:88)

]
Failure[
message='Worked ran into an unhandled exception'
type='Worker exception'
agentAddress=10.167.152.72
time=Fri Jun 13 18:16:44 UTC 2014
workerAddress=client:10.167.152.72
workerId=worker-10.167.152.72-4-client
test=TestCase{
id=MapTransactionTest
, class=com.hazelcast.stabilizer.tests.map.MapTransactionTest
, basename=MapTransactionTest
, keyCount=1000
, threadCount=3
}
cause=com.hazelcast.transaction.TransactionException: Transaction couldn't obtain lock for the key:966
at com.hazelcast.map.tx.TransactionalMapProxySupport.lockAndGet(TransactionalMapProxySupport.java:198)
at com.hazelcast.map.tx.TransactionalMapProxySupport.getForUpdateInternal(TransactionalMapProxySupport.java:104)
at com.hazelcast.map.tx.TransactionalMapProxy.getForUpdate(TransactionalMapProxy.java:88)
at com.hazelcast.map.client.AbstractTxnMapRequest.innerCall(AbstractTxnMapRequest.java:96)
at com.hazelcast.client.txn.BaseTransactionRequest.call(BaseTransactionRequest.java:40)
at com.hazelcast.client.CallableClientRequest.process(CallableClientRequest.java:27)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.processRequest(ClientEngineImpl.java:499)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:446)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.util.executor.PoolExecutorThreadFactory$ManagedThread.run(PoolExecutorThreadFactory.java:59)
at ------ End remote and begin local stack-trace ------.(Unknown Source)
at com.hazelcast.client.spi.impl.ClientCallFuture.resolveResponse(ClientCallFuture.java:160)
at com.hazelcast.client.spi.impl.ClientCallFuture.get(ClientCallFuture.java:109)
at com.hazelcast.client.spi.impl.ClientCallFuture.get(ClientCallFuture.java:90)
at com.hazelcast.client.txn.proxy.ClientTxnProxy.invoke(ClientTxnProxy.java:54)
at com.hazelcast.client.txn.proxy.ClientTxnMapProxy.getForUpdate(ClientTxnMapProxy.java:55)
at com.hazelcast.stabilizer.tests.map.MapTransactionTest$Worker$1.execute(MapTransactionTest.java:124)
at com.hazelcast.client.txn.ClientTransactionManager.executeTransaction(ClientTransactionManager.java:72)
at com.hazelcast.client.txn.ClientTransactionManager.executeTransaction(ClientTransactionManager.java:65)
at com.hazelcast.client.HazelcastClient.executeTransaction(HazelcastClient.java:299)
at com.hazelcast.client.HazelcastClientProxy.executeTransaction(HazelcastClientProxy.java:135)
at com.hazelcast.stabilizer.tests.map.MapTransactionTest$Worker.run(MapTransactionTest.java:120)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.stabilizer.tests.utils.ThreadSpawner$DefaultThread.run(ThreadSpawner.java:88)

]

ec2 Stabilizer hose box
/home/ec2-user/stabilizer-tests/final/xlarge/workers/1402675793387/worker-10.101.159.173-3-client/worker.log

FATAL 2014-06-13 18:14:43,635 [Thread-3] com.hazelcast.stabilizer.tests.utils.ExceptionReporter: Exception detected
com.hazelcast.core.OperationTimeoutException: No response for 120000 ms. Aborting invocation! InvocationFuture{invocation=BasicInvocation{ serviceName='hz:impl:mapService', op=TxnLockAndGetOperation{timeout=120000, thread=49}, partitionId=248, replicaIndex=0, tryCount=250, tryPauseMillis=500, invokeCount=1, callTimeout=60000, target=Address[10.237.187.71]:5701}, done=false} No response has been send backups-expected: 0 backups-completed: 0 reinvocations: 0
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.newOperationTimeoutException(BasicInvocation.java:782)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.waitForResponse(BasicInvocation.java:760)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:697)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676)
at com.hazelcast.map.tx.TransactionalMapProxySupport.lockAndGet(TransactionalMapProxySupport.java:196)
at com.hazelcast.map.tx.TransactionalMapProxySupport.getForUpdateInternal(TransactionalMapProxySupport.java:104)
at com.hazelcast.map.tx.TransactionalMapProxy.getForUpdate(TransactionalMapProxy.java:88)
at com.hazelcast.map.client.AbstractTxnMapRequest.innerCall(AbstractTxnMapRequest.java:96)
at com.hazelcast.client.txn.BaseTransactionRequest.call(BaseTransactionRequest.java:40)
at com.hazelcast.client.CallableClientRequest.process(CallableClientRequest.java:27)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.processRequest(ClientEngineImpl.java:499)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:446)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.util.executor.PoolExecutorThreadFactory$ManagedThread.run(PoolExecutorThreadFactory.java:59)
at ------ End remote and begin local stack-trace ------.(Unknown Source)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponse(BasicInvocation.java:862)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.resolveResponseOrThrowException(BasicInvocation.java:795)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:698)
at com.hazelcast.spi.impl.BasicInvocation$InvocationFuture.get(BasicInvocation.java:676)
at com.hazelcast.map.tx.TransactionalMapProxySupport.lockAndGet(TransactionalMapProxySupport.java:196)
at com.hazelcast.map.tx.TransactionalMapProxySupport.getForUpdateInternal(TransactionalMapProxySupport.java:104)
at com.hazelcast.map.tx.TransactionalMapProxy.getForUpdate(TransactionalMapProxy.java:88)
at com.hazelcast.map.client.AbstractTxnMapRequest.innerCall(AbstractTxnMapRequest.java:96)
at com.hazelcast.client.txn.BaseTransactionRequest.call(BaseTransactionRequest.java:40)
at com.hazelcast.client.CallableClientRequest.process(CallableClientRequest.java:27)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.processRequest(ClientEngineImpl.java:499)
at com.hazelcast.client.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:446)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.util.executor.PoolExecutorThreadFactory$ManagedThread.run(PoolExecutorThreadFactory.java:59)
at ------ End remote and begin local stack-trace ------.(Unknown Source)
at com.hazelcast.client.spi.impl.ClientCallFuture.resolveResponse(ClientCallFuture.java:160)
at com.hazelcast.client.spi.impl.ClientCallFuture.get(ClientCallFuture.java:109)
at com.hazelcast.client.spi.impl.ClientCallFuture.get(ClientCallFuture.java:90)
at com.hazelcast.client.txn.proxy.ClientTxnProxy.invoke(ClientTxnProxy.java:54)
at com.hazelcast.client.txn.proxy.ClientTxnMapProxy.getForUpdate(ClientTxnMapProxy.java:55)
at com.hazelcast.stabilizer.tests.map.MapTransactionTest$Worker$1.execute(MapTransactionTest.java:124)
at com.hazelcast.client.txn.ClientTransactionManager.executeTransaction(ClientTransactionManager.java:72)
at com.hazelcast.client.txn.ClientTransactionManager.executeTransaction(ClientTransactionManager.java:65)
at com.hazelcast.client.HazelcastClient.executeTransaction(HazelcastClient.java:299)
at com.hazelcast.client.HazelcastClientProxy.executeTransaction(HazelcastClientProxy.java:135)
at com.hazelcast.stabilizer.tests.map.MapTransactionTest$Worker.run(MapTransactionTest.java:120)
at java.lang.Thread.run(Thread.java:745)
at com.hazelcast.stabilizer.tests.utils.ThreadSpawner$DefaultThread.run(ThreadSpawner.java:88)
@enesakar
Copy link
Contributor

merging to #2516

This was referenced Jun 20, 2014
Closed
Closed
@mmedenjak mmedenjak added the Source: Internal PR or issue was opened by an employee label Jan 28, 2020
frant-hartm pushed a commit that referenced this issue Mar 26, 2021
@gierlachg
Provide error message for ClassNotFoundException while resolving ma…
873dd1e 
…ppings. (#2778)
@kulsagar325 kulsagar325 mentioned this issue May 19, 2021
Open
devOpsHazelcast pushed a commit that referenced this issue Jul 29, 2024
@orcunc @actions-user
Fix javadoc (#2778)
a87b311 
GitOrigin-RevId: f1e495e676cd9f194a19542674eca1ff8403f0d8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Source: Internal PR or issue was opened by an employee Team: Performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@pveentjer @enesakar @Danny-Hazelcast @mmedenjak