[serializer] Suggestions

[alphaqu]

Here come all of my suggestions to the serializer team.

1. Inline Methods

This is the biggest performance-wise. Inline all of the get methods. On serialization/Deseriliazation

readInt(bytes, pos)

pos = this.pos;
bytes[pos++] = value
bytes[pos++] = value << 8
bytes[pos++] = value << 16
bytes[pos++] = value << 24
this.pos = pos;

2. Compress Booleans / Nullable

The Bool/Null currently takes a byte, which in this case can actually store 4 states of it.
Example

Null | Null | NotNull | Null
[1,0,0,0][1,0,0,0][0,0,0,0][1,0,0,0]

Null | Null | NotNull | Null
[1,1,0,1]

3. Cast booleans

If you inline the methods, you have the opportunity to abuse the jvm, In this case, javac does not allow booleans to be cast to bytes. But at runtime, this is fully allowed as booleans are stored internally as bytes.

bytes[pos++] = bool ? 1 : 0
bool = bytes[pos++] == 1

bytes[pos++] = (byte)bool
bool = (boolean)bytes[pos++];

4. Overhaul the SerializerBuilder

Please, it's 1300 lines, I made a more advanced one that can follow on with types through the class in 200 lines with clear and clean code, if you want the code just mention it in the reply.

5. Maven

Why do you guys use maven instead of Gradle? Gradle is so much faster on builds and is so much more powerful, I already have a fork that allows for basic compiling, if you guys want I can set up the repository plugin.

6. Subproject

Also, why do you guys have all of these amazing projects in a single repository, I suggest splitting the main ones into multiple projects so it's easy to manage all of the issues and pr.s

[eduard-vasinskyi]

1. Inline Methods

According to our measurements, JVM does a pretty good job of inlining hot methods. In fact, having large monolithic methods with too aggressive inlinings could severely impede the performance of generated machine code. For example, it makes it difficult for JVM to allocate local variables to registers.

Also, aggressive inlining makes it impossible to implement certain features like support of cyclic data structures.
That’s why we had to design the serializer trying to maintain a perfect balance between the code which is inlined and which is not (the Codegen expressions naturally result in “inlined” code, so we had to extract some code into standalone methods).
Of course, there could be suboptimal cases, so you are welcome to provide us with patches / specific examples if you spot them.

2. Compress Booleans / Nullable

All primitives are aligned to a 1-byte boundary, for simplicity and performance reasons
(but you can easily change that by implementing your own setters/getters, which packs your app-specific fields into byte/int/long the way you want it).

But you are right, unlike other nullable data types (strings, arrays, collections, enums) which have efficient binary representation for their null values, nullable booleans were not optimized.

We are introducing a new binary Compatibility Level 4 to address this issue, it will be part of ActiveJ 5.0 beta3+/rc.

3. Cast booleans

This is a good idea! We will try to implement it.

4. Overhaul the SerializerBuilder

It has lots of features like annotation support, pluggable serializers for user-definable data types, binary compatibility levels, versioning, and ordering of data fields, in fact, 1/3 of its code consists of just setters and configuration.

Also, in ActiveJ 5.0+, SerializerBuilder has already been fully redesigned, please take a look if you have not. Unlike ActiveJ 4.0, now SerializerBuilder is based on powerful TypeScannerRegistry in its core, which is a very abstract and universal way to traverse generic type trees and reduce them into user-defined result type, using user-defined transformation rules (like directly transforming Map<String, @SerializerNullable List<Integer>> into corresponding Codegen expressions).

Of course, it’s possible to strip all those features and just hardcode everything, but our production needs proved that all those features are generally a must-have.

5. Maven

We hate Maven just like you do, but it's well supported by IDEs and works ok for now.

6. Subproject

It’s easier to refactor it that way when changes in one component propagate to others. You can use only the components you need, they are highly modular with clean minimalistic dependencies between each other.

[eduard-vasinskyi]

Hi, @alphaqu

ActiveJ v5.0-rc3 has been released. We have implemented some of your suggestions.

Now, nullable booleans are serialized as a single byte. This is available when using compatibility level 4 (a new default) in the serializer builder.

We also cast booleans to bytes and vice-versa when serializing boolean values.

A serializer overall was refactored and simplified, now it provides more flexible bytecode caching capabilities.

Also, this version has already been tested in our production applications, so a final ActiveJ v5.0 is planned to be released in the next few days.