Environmentally sensitive generated values for serialVersionUID and class serialization #24

Closed
alkemist opened this Issue Oct 24, 2013 · 22 comments

Projects

None yet

5 participants

@alkemist

The Groovy Remote Control allows Groovy closures to be serialized and injected into a running system. This relies on shipping serialized Java objects and .class files.

When Spring Loaded is active in either of the JVMs involved, this does not work.

If the client (JVM sending the closure) has Spring Loaded enabled, a NPE will be thrown on the server side when the closure object is deserialized.

If the server has Spring Loaded enabled, a failure like the following will occur:

Caused by: java.io.InvalidClassException: Script$_run_closure1; local class incompatible: stream classdesc serialVersionUID = -3252672982768198903, local class serialVersionUID = 1362528599794029384

A similar failure will occur if both JVMs have Spring Loaded enabled.

I have a test project that can be used for running in such a setup.

https://github.com/alkemist/remote-control-springloaded

@jdbeutel

The NPE is not appropriate, because the API explicitly allows null for the class name.

@fcamblor
fcamblor commented Nov 6, 2013

👍 on this. Encountered the same problem when trying to deserialize some POJO after enabling spring-loaded on my project (POJO was not serialized using spring-loaded).

I suppose this is due to bytecode instrumentation... shouldn't this instrumentation keep the original serial version uid ?
Or at least allow to enable/disable this behaviour with a flag ?

Note : I didn't defined any serial version uuid on my POJO, relying on the default uuid generation.

@alkemist alkemist referenced this issue in alkemist/grails-remote-control Feb 13, 2014
Closed

Not working with Grails 2.3 #12

@alkemist

Any update on this?

@antonmos

An update would be most welcome!

@aclement
Contributor
aclement commented Mar 3, 2014

Unfortunately serialization is an area I haven't spent much time in yet. My latest work was really focused on reloading Java8 code (lambdas). I will try to find a bit of time to look into this. Simply generating serialversionUID may not be enough to make everything behave nicely.

@aclement
Contributor
aclement commented Mar 6, 2014

Looking at this now, using the Groovy Remote Control codebase (It was easy to recreate the error using that, thanks!). My first thought is that the reflective calls in java lang ObjectStream are not being rewritten, so I'm addressing that. System classes containing reflective calls that need rewriting are currently whitelisted in the agent rather than us hunting through all system classes (which would slow startup even more).

@aclement
Contributor

I have some test cases relating to this passing now. I need to tidy it all up. Two things needed sorting out:

  • ObjectStreamClass needed reflective interception
  • ObjectStreamClass has a native method 'hasStaticInitializer' whose return value does influence serialization. All reload able types have a static initializer so we need to intercept this too - but that isn't totally trivial because it is native.

I can now serialize objects outside of a system running with the agent and then deserialize them inside.

There may be follow on work to handle objects that have new fields added and then get serialized, but I'll get this first bit of work committed for now.

@antonmos

Are there any plans to release this fix? Thanks

@aclement
Contributor

All committed a while ago, should be in 1.2.0 snapshots already. I'd be interested in any feedback.

@antonmos

I tried to integrate the snapshot with Alkemist's test project and unfortunately still getting the serialization error.
Did it work for you?
i am using java version "1.7.0_45" on a Mac

Caused by: java.io.InvalidClassException: Script$_run_closure1; local class incompatible: stream classdesc serialVersionUID = 5042741366753642612, local class serialVersionUID = 1362528599794029384
    at java_io_ObjectInput$readObject.call(Unknown Source)
    at groovyx.remote.server.CommandInvoker.instantiate(CommandInvoker.groovy:61)
    at groovyx.remote.server.CommandInvoker.invokeAgainst(CommandInvoker.groovy:37)
    at groovyx.remote.server.CommandInvoker$invokeAgainst.call(Unknown Source)
    at groovyx.remote.server.CommandChainInvoker.invokeAgainst(CommandChainInvoker.groovy:37)
    at org.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1254)
    at groovyx.remote.server.CommandChainInvoker.invokeAgainst(CommandChainInvoker.groovy)
    at groovyx.remote.server.CommandChainInvoker$invokeAgainst.call(Unknown Source)
    at groovyx.remote.server.Receiver.invokeCommandChain(Receiver.groovy:129)
    at org.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1254)
    at groovyx.remote.server.Receiver.execute(Receiver.groovy:125)
    at groovyx.remote.server.Receiver$execute.call(Unknown Source)
    at groovyx.remote.transport.http.RemoteControlHttpHandler.doExecute(RemoteControlHttpHandler.groovy:84)
    at org.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1254)
    at groovyx.remote.transport.http.RemoteControlHttpHandler$_handle_closure1.doCall(RemoteControlHttpHandler.groovy:38)
    at org.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1254)
    at groovyx.remote.transport.http.RemoteControlHttpHandler.handle(RemoteControlHttpHandler.groovy:37)
    at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77)
    at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:80)
    at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77)
@aclement
Contributor

That test program did work fine for me - but I'll admit I wasn't using a downloaded snapshot I was using a local build. Let me retry and make sure the snapshot on the server contains the change.

@aclement
Contributor

I just tried it again with the 1.2.0 build snapshot and it worked. I run it with:

./gradlew runScript

I got it to download the right version by editing the dependency to:

springloaded "org.springframework:springloaded:1.2.0.BUILD-SNAPSHOT"

(note the change in groupId)

I then have no idea how to alter the -javaagent line to automagically point to what was downloaded so I modified it to point into the .gradle repo on my disk where the downloaded jar had been placed:

"-javaagent:/Users/aclement/.gradle/caches/artifacts-26/filestore/org.springframework/springloaded/1.2.0.BUILD-SNAPSHOT/jar/3665292afd3f2fc9822f26416ff00488188917dd/springloaded-1.2.0.BUILD-SNAPSHOT.jar"

./gradlew runScript
:compileJava UP-TO-DATE
:compileGroovy UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:startApp
server: started
:compileTestJava UP-TO-DATE
:compileTestGroovy UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:runScript
:stopApp

BUILD SUCCESSFUL

@aclement
Contributor

I was using Java8 (build 128) and Java7 u51

@antonmos

Turns out the snapshot jar referenced in the https://github.com/spring-projects/spring-loaded/blob/master/README.md is not up to date. It is working now with the jar from the repo you used. Thanks!

@aclement
Contributor

I did wonder if that might be it. Glad it is working for you now, I'll do something about the read me.

@aclement aclement closed this Mar 22, 2014
@aclement aclement added this to the 1.2.0 milestone Mar 22, 2014
@aclement aclement self-assigned this Mar 22, 2014
@antonmos

Unfortunately, it is not working on java1.6_29. Is it possible to support it?

@antonmos

Btw, here is how to get gradle to give you location of the snapshot jar:

def springloadedJar = configurations.springloaded.files.find { it.name.startsWith("springloaded")}

    agentArgs <<
            "-javaagent:$springloadedJar.absolutePath" <<
            "-noverify"
@aclement
Contributor

No doubt the implementation of serialization changed between java6 and the version I was looking at. Sometimes I find it is just a change in the names of fields that breaks things and we can change the patch to just 'try both' (the old and the new). Compare ObjectStreamClass between that version and the latest?

@antonmos

Will you reopen this issue or should I log a new one?

@aclement
Contributor

Let's have a new issue. Have you tried the most recent Java6 service refresh?

@antonmos

Reproduced on 1.6_65 and opened Issue #54. Thanks!

@antonmos
antonmos commented Apr 1, 2014

Just so I can plan my work, do you plan to pick up Issue #54 in the near future? Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment