-
Notifications
You must be signed in to change notification settings - Fork 96
Serialization FAQ
A: All classes implementing either Serializable, Externalizable, Collection or Map interface can be serialized.
Technically all other classes also could be, but the limitation is introduced intentionally to decrease the possibility of mistakes.
A: The type of the field does not matter. What makes sense is the type of the specific object residing in this field. If it is serializable, then everything is OK.
A: serialVersionUID is simply ignored in one-nio. It can be safely removed.
A: Unique ID (UID) is given not to the class, but to the serializer. One class can have multiple serializers: one native (built from the local version of the class), and any number of external serializers (received from other servers with different version of this class).
UID is calculated as a hash function of names, types and the order of instance fields.
A: All non-static and non-transient fields, just like in standard Java serialization.
Q: Does a class need to have a public constructor or a default constructor in order to be deserializable?
A: No. No constructor is called during deserialization, except for collections and maps (see below).
A: There is limited support for readObject/writeObject. These methods will be called, but they should not work with the stream directly. The only stream methods they may call are defaultReadObject and defaultWriteObject. Other calls will result in exception.
readObject/writeObject may be used for data preparation, e.g. for filling transient fields after deserialization. For more flexible serialization use Externalizable interface.
A: Yes. Externalizable is completely supported.
A: Please update JDK to version 6u26 or newer.
A: All fields of these classes are transient, therefore our serialization does not work. Please use java.util.Date or java.sql.Timestamp which have dedicated serializers in one-nio.
A: There difference in footprint is minimal: wrappers have one extra byte for the header. For example, long occupies 8 bytes while Long will take 9 bytes.
There will be a slight difference in speed, so, if you don’t expect nulls, the primitives are preferred.
A:
- Addition of a new field (which will be initialized with 0 or null);
- Field removal (received value for this field will be ignored);
- Field reordering;
- Field type changes (see below);
- Field renaming (when annotated with
@Renamed).
A:
class MyClass {
@Renamed(from = "oldName")
String newName;
}
(!) It is not recommended to mix field renaming with other modifications in a single software release deployment.
(i) After successful deployment of a new version, as soon as all servers and clients obtain the class with the field renamed, this annotation can be removed.
A: You can do this, if you add @Renamed annotation with the fully qualified old name:
package one.comp.somepackage;
@Renamed(from = "one.comp.somepackage.OldClass")
class NewClass { ... }
(!) Do not rename classes occuring in signatures of remote methods. Otherwise there will be no backward compatibility of the remote service.
(i) After successful deployment of a new version, as soon as all servers and clients obtain the class with the field renamed, this annotation can be removed.
A: Yes. The @Renamed annotation is required in this case.
package one.comp.newpackage;
@Renamed(from = "one.comp.oldpackage.MovedClass")
class MovedClass { ... }
Q: Is it allowed to change the class hierarchy? Will the old class MyClass be compatible with the new MyClass extends NewParent or MyClass implements NewInterface?
A: Yes, these kinds of changes are allowed.
A: No. It is possible to add, remove and rename methods without losing backward compatibility in respect to serialization.
A: The new method will be incompatible with an old one. In order to do such kind of refactoring, create a new method (with different name or signature) without removing an old one. After an successful deployment of a new version, when the old method is not used any more, it can be safely removed. We will probably add @Renamed annotation support for methods in the future.
A: You can do so in two cases:
- if the method was
void, or if it always returnednull; - if the new type and the old type belong to the same class hierarchy, i.e. one is the parent of another.
A: Yes, it is possible to add a field of a new class. While it is null, both sides can communicate well, even if one side has no this class locally. However as soon as the real instances of the new class start to travel over remoting, ClassNotFoundException will be thrown.
Q: The type of the field has changed and the name remained the same. How this will affect serialization?
A: It depends. The old type will be automatically converted to the new type in the following cases:
- The new type is the parent of the old type. No conversion is needed.
- The new type is inherited from the old type. The type cast will occur. If the type cast fails,
ClassCastExceptionwill be thrown. - A primitive type -> another primitive type. The value will be converted according to general Java language rules, e.g. (long) intValue, (byte) shortValue etc.
booleancan also be converted to integer types: false <-> 0, true <-> 1. - A primitive type -> the wrapper (e.g. long -> Long). This is a special case of #7:
Long.valueOf(long). - The wrapper -> a primitive (e.g. Integer -> int). This is a special case of #8:
Integer.intValue(). - Number -> Number (e.g. Integer -> Long). The value will be converted through the intermediate primitive type.
- Invocation of
public static NewClass valueOf(OldClass), if the method exists. - Invocation of
public NewClass anyMethodName(), if the method exists.
(i) If none of the rules above can be applied, no conversion is made. The changed field will be initialized with the default value: 0 or null.
A: Yes, this is a compatible change. Anyway make sure there are no two fields with the same name.
A: There is a special trick for such a case. You need to call the method Repository.addInlinedClass("full.name.of.Holder") at the application initialization time. Then all fields of the Holder class will be inlined directly to the stream, so that the binary representation remains unchanged. Therefore the refactoring will be backward compatible.
Q: How is enum serialized? Is it allowed to change constant names, add, remove or reorder constants?
A: enums are encoded with 2-byte integer during serialization. one-nio does its best to resolve possible modifications:
- if a client and a server have enum constants reordered, the correct values will be matched by the constant names;
- if a constant is renamed but its position remained the same, the values will also be matched correctly;
- if one side receives a constant which it does not know, it will be silently replaced with null.
(!) To avoid conflicts, do not rename and reorder constants together within a single new release deployment.
A: Classes implementing Collection interface are serialized like an array. The fields of the class do not matter.
The class should have a public default constructor in order to be deserialized. If there is no such a constructor, the class will be deserialized as an ArrayList, LinkedList, HashSet or a TreeSet, depending on implemented interfaces.
A: Classes implementing Map interface are serialized as key-value pairs. The fields of the class do not matter.
The class should have a public default constructor in order to be deserialized. If there is no such a constructor, the class will be deserialized as a HashMap or a TreeMap, depending on whether it implements SortedMap interface.
Q: Where do I find if there are any serialization problems related to different versions of my application classes?
A: All detected version conflicts are printed to one.nio.serial.Repository logger. Look for the messages like
WARN [one.nio.serial.Repository] [11d88aa11ca2e665] Local field is missed in stream: java.lang.Long one.comp.spam.ClientInfoMetadataImpl.currentUserId WARN [one.nio.serial.Repository] [13f21415e8cd1dd3] Stream field is missed locally: int odkl.index.query.ResultItem.audience WARN [one.nio.serial.Repository] Missed local enum constant one.image.server.persistence.ImageType.PROFILE_320 WARN [one.nio.serial.Repository] Trying to serialize anonymous class: one.comp.groupphoto.dcache.DGroupPhotoCache$1
There is also serialization statistics available through JMX MBean one.nio.serial:type=Serialization
A: Run the JVM with the option -Done.nio.gen.dump=/path/to/generated/files
You’ll find the generated .class files in the given directory.
A: Slides - http://www.slideshare.net/AndreiPangin/pangin-highload
Presentation - http://www.youtube.com/watch?v=gIh0X-RkftY