Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scripting: Wrap groovy script exceptions in a serializable Exception object #6598

Closed
nik9000 opened this issue Jun 23, 2014 · 7 comments · Fixed by #6628

Comments

@nik9000
Copy link
Contributor

commented Jun 23, 2014

I'm playing around with groovy and I think exceptions aren't serializing properly:

[2014-06-23 18:19:54,091][INFO ][org.elasticsearch.index.mapper] Action Failed
org.elasticsearch.transport.RemoteTransportException: [node_0][inet[/192.168.0.101:9300]][index]
Caused by: org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream
Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize exception response from stream
    at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:169)
    at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.InvalidClassException: failed to read class descriptor
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1603)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
    at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1483)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1333)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
    at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:500)
    at java.lang.Throwable.readObject(Throwable.java:914)
    at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
    at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:500)
    at java.lang.Throwable.readObject(Throwable.java:914)
    at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
    at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:500)
    at java.lang.Throwable.readObject(Throwable.java:914)
    at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
    at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:167)
    ... 23 more
Caused by: java.lang.ClassNotFoundException: Script4
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at org.elasticsearch.common.io.ThrowableObjectInputStream.loadClass(ThrowableObjectInputStream.java:93)
    at org.elasticsearch.common.io.ThrowableObjectInputStream.readClassDescriptor(ThrowableObjectInputStream.java:67)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
    ... 62 more

I'm in the middle of work on #6566 so I don't have easy reproduction steps, but I'll see if I can make some soon.

@nik9000

This comment has been minimized.

Copy link
Contributor Author

commented Jun 23, 2014

Ping @dakrone. I'll work on it in a bit but I think it has to do with when a variable isn't found.

@dakrone

This comment has been minimized.

Copy link
Member

commented Jun 23, 2014

@nik9000 sounds good, I definitely want to figure out what's causing this.

@nik9000

This comment has been minimized.

Copy link
Contributor Author

commented Jun 23, 2014

@dakrone, got it: start a bunch of servers. Enough that your request goes to more then one. Then do this:

curl -XPOST "http://localhost:9200/test/test/1?pretty" -d '{"content": "findme"}'
curl -XPOST "http://localhost:9200/test/test/2?pretty" -d '{"title": "cat", "content": "findme"}'
curl -XPOST "http://localhost:9200/test/test/3?pretty" -d '{"title": "table", "content": "findme"}'
curl -XPOST "http://localhost:9200/test/_refresh?pretty"
curl -XPOST "http://localhost:9200/test/test/_search?pretty" -d '{
    "query": {
        "filtered": {
            "filter": {
                "script": {
                    "script": "1 == not_found",
                    "lang": "groovy"
                }
            }
        }
    }
}'

I don't imagine the contents of the documents matter - just that they end up on a bunch of nodes.
Here is what that spits out for me:

{
  "took" : 35,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 2,
    "failed" : 3,
    "failures" : [ {
      "index" : "test",
      "shard" : 4,
      "status" : 500,
      "reason" : "QueryPhaseExecutionException[[test][4]: query[filtered(ConstantScore(ScriptFilter(1 == not_found)))->cache(_type:test)],from[0],size[10]: Query Failed [Failed to execute main query]]; nested: MissingPropertyException[No such property: not_found for class: Script5]; "
    }, {
      "index" : "test",
      "shard" : 3,
      "status" : 500,
      "reason" : "RemoteTransportException[Failed to deserialize exception response from stream]; nested: TransportSerializationException[Failed to deserialize exception response from stream]; nested: InvalidClassException[failed to read class descriptor]; nested: ClassNotFoundException[Script3]; "
    }, {
      "index" : "test",
      "shard" : 2,
      "status" : 500,
      "reason" : "RemoteTransportException[Failed to deserialize exception response from stream]; nested: TransportSerializationException[Failed to deserialize exception response from stream]; nested: InvalidClassException[failed to read class descriptor]; nested: ClassNotFoundException[Script2]; "
    } ]
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : [ ]
  }
}
@nik9000

This comment has been minimized.

Copy link
Contributor Author

commented Jun 23, 2014

3 servers did it for me.

@kimchy

This comment has been minimized.

Copy link
Member

commented Jun 24, 2014

I think it comes from trying to serialize the groovy exceptions objects, I would suggest we catch a script execution exception, but not have the throwable in our wrapping script exception. Users might run client side node/transport clients that don't have groovy in the class path for example.

@nik9000

This comment has been minimized.

Copy link
Contributor Author

commented Jun 24, 2014

I think @kimchy's right. What I'm seeing is that groovy decided to name the compiled classes something different but even without that the chance that the user doesn't even have groovy in their classpath means we should transform the exception without adding the cause. I imagine MVEL didn't have this problem because MVEL was required and was interpreted instead of compiled to byte code.

@dakrone

This comment has been minimized.

Copy link
Member

commented Jun 30, 2014

Thanks for bringing this up @nik9000!

@dakrone dakrone changed the title Groovy exeptions blow up Scripting: Wrap groovy script exceptions in a serializable Exception object Jul 7, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.