Encountered unregistered class ID

[idar]

Hi

I get this exception when sending messages across two nodes.
we use deault strategy and we have logged that kryo uses writeClassAndObject and readClassAndObject.

This is in a akkacluser and there are messages going back and fort with java serialization and google protbuf.

any ideas?

AkkaConfig:
actor {
provider = "akka.cluster.ClusterActorRefProvider"

     serializers {
         java = "akka.serialization.JavaSerializer"
         proto = "akka.remote.serialization.ProtobufSerializer"
         kryo =  "com.romix.akka.serialization.kryo.KryoSerializer"
     }

     serialization-bindings {
         "no.ks.eventstore2.Event" = kryo
     }

     kryo{
        type = "graph"
        idstrategy = "default"
        serializer-pool-size = 16
        buffer-size = 4096
        use-manifests = false
        implicit-registration-logging = true
        kryo-trace = false
        kryo-custom-serializer-init = "no.ks.KryoInit"
     }
}

extensions = ["com.romix.akka.serialization.kryo.KryoSerializationExtension$"]

exception:
com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 720389982
at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:767)
at com.romix.akka.serialization.kryo.KryoBasedSerializer.fromBinary(KryoSerializer.scala:279)
at com.romix.akka.serialization.kryo.KryoSerializer.fromBinary(KryoSerializer.scala:135)

[idar]

00:29 TRACE: [kryo] Register class ID 1028558732: no.ks.svarut.bruker.BrukerOpprettet (com.esotericsoftware.kryo.ser
ializers.FieldSerializer)
Implicitly registered class with id: no.ks.svarut.bruker.BrukerOpprettet=1028558732

Its my classes that get these ids.

[romix]

Can you enable
kryo-trace = false
and see how this class is registered and which ID is assigned to it?
BTW, do you define "mappings" or "classes" sections, as described in the README?

[romix]

I updated README to be more explicit about initialization of Kryo and assigning the same class IDs on both sides

[idar]

I wanted to use the default method, without specifying mappings or classes. Let kryo write classnames and object.

[romix]

One important thing to realize is:
If you do not follow the guidelines from a readme, Kryo randomly assigns some IDs to classes. It doesn't matter if the same instance of Kryo is used for both serialization and deserialization. But if two different instances of Kryo are used (which is the case when you use actors and they run on different JVMs or different ActorSystems), then you have to ensure that the same IDs are used for the same classes. Otherwise the receiving side cannot deserialize properly. Readme describes how to ensure that the same IDs are used. You either use "classes" or "mappings" or custom initializer which invokes register for a typical object. The later approach is probably the easiest, as it will recursively register all classes used by a typical object and its sub-objects. And since this custom initializer will be invoked on both sides it will ensure that the same IDs are used on both sides.

[idar]

Isn't it therefore kryo writes the complete class name in the data, when calling writeClassAndObject? Thats what we use when we write classes to our database with kryo. No registration og classes.
idstrategy=default, should make classes and mappings ignored right?

[romix]

Not quite. The story goes like this:
If the class is not registered at all and Kryo sees it for the first time, it would write ONCE the classname and the assigned id and later it would simply use the assigned class id for any subsequent object of the same class and would not write a class name. The deserializing side would read the serialized representation, discover the FIRST occurence of class, read the classname, will try to assign the same id to it and then proceed.

But if your deserializer (e.g. a receiving actor) was receiving some other messages before it could happen that it has seen some other classes in the meantime and assigned the same IDs to them. And now, when it receives something with your class those IDs are occupied already.

Therefore, I really suggest that you initialize Kryo exactly in the same way on all sides.

BTW, the reason why Kryo always writes classnames into a database for you is probably due to the fact, that you create a new Kryo instance every time you serialize. This way, for this fresh Kryo instance your class is always a fist time :-) But in this library, as it is supposed to be a high-performance one, we do pooling of Kryo instances, because creating Kryo instances for each operation is too expensive.

I really suggest you try to register your typical objects in your custom initializer. It is easy (because you do not need to explicitly enumerate all classes as it is the case with "mappings" and "classes" sections) and it will automatically register all classes used by the object graph represented by this object and its sub-objects.

[idar]

I think it just wrote the classname the first time to the database. We read the classes in sequence every time :)

I will have to register all the classes, thanks for all the help.

[romix]

No problem. Let me know if it works for you.

[idar]

Now using explicit registration and my kryo-init class for registering classes. Seems to work perfectly. I had huge performance issues with networking until i found the throughput parameter on the default dispatcher. Throughput is now 10000.

[romix]

Very cool. Can you quantify how much faster it is for your use-cases? BTW, if you want to make it even faster, the Unsafe-based Input/Output streams in 2.22-SNAPSHOT can give a significant performance boost.

BTW, it would be nice if you could report on the Akka mailing list that it works for you now and what kind of performance improvement you got. This could be useful to promote this library :-)

[romix]

Just saw your comment on the mailing list. Thanks a lot!

[idar]

5-6 min join time in my cluster when i burst messages across. With throuput 10000 its down to 2min. Profiling shows insignificant time in kryo. Its akka and networkstack that uses cputime.

[mklosi]

im registering all my classes upon serialization with scala i have "serialization required" true. and it serializes fine. but when deserializing on another machine that runs java, i get the
com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 113

im trying to figure out how to use the custom initializers and give them a try.