ISPN-6680 InterruptedException during PutAll from Hot Rod client prev… #4360

gustavocoding · 2016-05-19T15:18:48Z

…ents failover

https://issues.jboss.org/browse/ISPN-6680
https://issues.jboss.org/browse/ISPN-6692

gustavocoding · 2016-05-23T08:19:51Z

Adding ISPN-6692 as well

danberindei · 2016-05-25T08:15:19Z

server/hotrod/src/main/scala/org/infinispan/server/hotrod/OperationStatus.scala

@@ -30,6 +30,7 @@ object OperationStatus extends Enumeration {
   val OperationTimedOut = Value(0x86) // todo: test
   val NodeSuspected = Value(0x87)
   val IllegalLifecycleState = Value(0x88)
+   val Interrupted = Value(0x89)


Unless you actually need a different behaviour from IllegalLifecycleState, I would prefer to see a single ServerNotAvailable operation status covering both. I know sometimes these exceptions can sometimes come from another node, but it should be the core's job to retry them and/or translate them to SuspectException.

My initial though was to piggyback on IllegalLifecycleState, but an InterruptedException could be throw in other situations apart from server going down. For example, a long running job (script?) that gets explicitly cancelled by the user

gustavocoding · 2016-05-25T13:06:23Z

@danberindei thoughts? TBH I was not willing to bind InterruptedException to a server stopping, do you think it should?

danberindei · 2016-05-25T13:10:08Z

@gustavonalle If the server isn't stoppping, and a long task is being cancelled, should the client retry it? Could the server send back an IllegalLifecycleState status if it caught an InterruptedException and the server is stopping, and pass the InterruptedException unmodified otherwise?

gustavocoding · 2016-05-25T13:49:30Z

If the server isn't stoppping, and a long task is being cancelled, should the client retry it?

IMO no, as It could have been explicitly canceled (maybe the script or task was misbehaving).
Note that in this PR it is not retrying the ExecuteOperation upon InterruptedExceptions

Could the server send back an IllegalLifecycleState status if it caught an InterruptedException and the server is stopping, and pass the InterruptedException unmodified otherwise?

It makes sense

gustavocoding · 2016-06-13T08:26:17Z

@danberindei I've simplified the PR, without changing Hot Rod version, and wrapping InterruptedException inside an IllegalLifecycleException.
Reasons for this is the need to have this backported to several other branches and that we currently don't have explicitly "interruptible" operations in the server. We can review the need to expose cancelations to the client later.

gustavocoding · 2016-06-28T14:43:51Z

rebased

danberindei · 2016-07-05T16:04:24Z

client/hotrod-client/src/main/java/org/infinispan/client/hotrod/logging/Log.java

@@ -226,8 +226,8 @@
   IllegalStateException queryParameterNotSet(String filterConverterFactoryName);

   @LogMessage(level = WARN)
-   @Message(value = "Server not reachable whilst closing iteration '%s'", id = 4061)
-   void ignoringServerUnreachable(String iterationId);
+   @Message(value = "Ignoring error when closing whilst closing iteration '%s'", id = 4061)


You have to choose between when and whilst :)

I've removed whilst closing myself and pushed to master. Trying to rebase on 8.2.x...

danberindei · 2016-07-05T19:42:32Z

@gustavonalle after rebasing to 8.2.x I got a failure in DistTopologyChangeUnderLoadSingleOwnerTest.testRestartServerWhilePutting

java.util.concurrent.ExecutionException: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=453 returned server error (status=0x85): java.lang.NullPointerException

And this exception in the log:

19:47:43,569 DEBUG (HotRodServerWorker-119-1:[]) [HotRodExceptionHandler] Exception caught
java.lang.NullPointerException
    at org.infinispan.server.hotrod.ContextHandler.channelActive(ContextHandler.java:210) ~[infinispan-server-hotrod-8.2.3-SNAPSHOT.jar:8.2.3-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:168) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:154) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:64) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:168) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:154) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:64) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:168) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:154) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:64) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at org.infinispan.server.core.transport.StatsChannelHandler.channelActive(StatsChannelHandler.java:45) [infinispan-server-core-8.2.3-SNAPSHOT.jar:8.2.3-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:168) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:154) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:933) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:458) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:370) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:420) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:280) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) [netty-all-4.0.36.Final.jar:4.0.36.Final]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]

Closing this PR, please open a separate one for 8.2.x.

gustavocoding · 2016-07-12T08:27:11Z

@danberindei #4450 for 8.2.x

gustavocoding added the Ready for Review label May 19, 2016

danberindei reviewed May 25, 2016
View reviewed changes

danberindei added Changes Suggested and removed Ready for Review labels May 25, 2016

gustavocoding force-pushed the ISPN-6680 branch from 068744b to 3062bd9 Compare June 13, 2016 08:19

gustavocoding assigned danberindei Jun 13, 2016

gustavocoding added Ready for Review Urgent! Backport and removed Changes Suggested labels Jun 13, 2016

gustavonalle added 3 commits June 28, 2016 15:42

ISPN-6692 Handle exceptions in RemoteIterator close

eaf16e5

ISPN-6680 Handle InterruptedException in the client

161efd3

ISPN-6680 Handle InterruptedException in the server

0a1afce

gustavocoding force-pushed the ISPN-6680 branch from 3062bd9 to 0a1afce Compare June 28, 2016 14:43

danberindei reviewed Jul 5, 2016
View reviewed changes

danberindei closed this Jul 5, 2016

gustavocoding deleted the ISPN-6680 branch February 20, 2018 12:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ISPN-6680 InterruptedException during PutAll from Hot Rod client prev… #4360

ISPN-6680 InterruptedException during PutAll from Hot Rod client prev… #4360

gustavocoding commented May 19, 2016 •

edited

gustavocoding commented May 23, 2016

danberindei May 25, 2016

gustavocoding May 25, 2016

gustavocoding commented May 25, 2016

danberindei commented May 25, 2016

gustavocoding commented May 25, 2016

gustavocoding commented Jun 13, 2016 •

edited

gustavocoding commented Jun 28, 2016

danberindei Jul 5, 2016

danberindei Jul 5, 2016

danberindei commented Jul 5, 2016

gustavocoding commented Jul 12, 2016

ISPN-6680 InterruptedException during PutAll from Hot Rod client prev… #4360

ISPN-6680 InterruptedException during PutAll from Hot Rod client prev… #4360

Conversation

gustavocoding commented May 19, 2016 • edited

gustavocoding commented May 23, 2016

danberindei May 25, 2016

Choose a reason for hiding this comment

gustavocoding May 25, 2016

Choose a reason for hiding this comment

gustavocoding commented May 25, 2016

danberindei commented May 25, 2016

gustavocoding commented May 25, 2016

gustavocoding commented Jun 13, 2016 • edited

gustavocoding commented Jun 28, 2016

danberindei Jul 5, 2016

Choose a reason for hiding this comment

danberindei Jul 5, 2016

Choose a reason for hiding this comment

danberindei commented Jul 5, 2016

gustavocoding commented Jul 12, 2016

gustavocoding commented May 19, 2016 •

edited

gustavocoding commented Jun 13, 2016 •

edited