Android - Legacy Type Mapping works different from JDK

[zdenek-jonas]

I have two classes:

public class ByteLegacy {

    private byte byteTo_float;
    private byte byteTo_double;
    private byte byteTo_short;
    private byte byteTo_long;
    private byte byteTo_byte;
    private byte byteTo_int;
    private byte byteToString;
    private byte byteTo_boolean;
    private byte byteTo_char;
    private byte byteToByte;
    private Byte copyByteTo_byte;


    public static ByteLegacy fillSample() {
        ByteLegacy legacy = new ByteLegacy();

        legacy.byteTo_float = '1';
        legacy.byteTo_double = '2';
        legacy.byteTo_short = '3';
        legacy.byteTo_long = '4';
        legacy.byteTo_byte = 'a';
        legacy.byteTo_int = '6';
        legacy.byteToString = '5';
        legacy.byteTo_boolean = '0';
        legacy.byteTo_char = 'c';
        legacy.byteToByte = 'x';
        legacy.copyByteTo_byte = 'i';
        return legacy;
    }
}

public class ByteLegacy2 {

    private float byteTo_float;
    private double byteTo_double;
    private short byteTo_short;
    private long byteTo_long;
    private byte byteTo_byte;
    private int byteTo_int;
    private char byteToString;
    private boolean byteTo_boolean;
    private char byteTo_char;
    private Byte bytToByte;
    private byte copyByteTo_byte;
}

I save the first one, change the class and start with the second class
A received:

ByteLegacy{byteTo_float=49, byteTo_double=50, byteTo_short=51, byteTo_long=52, byteTo_byte=97, byteTo_int=54, byteToString=53, byteTo_boolean=48, byteTo_char=99, byteToByte=120, ByteTo_byte=105}

ByteLegacy2{byteTo_float=0.0, byteTo_double=0.0, byteTo_short=49, byteTo_long=0, byteTo_byte=97, byteTo_int=50, byteToString=5, byteTo_boolean=true, byteTo_char=c, bytToByte=120, copyByteTo_byte=105}

This is not an conversion what i expect.
On the classic JVM works as intended:

ByteLegacy{byteTo_float=49, byteTo_double=50, byteTo_short=51, byteTo_long=52, byteTo_byte=97, byteTo_int=54, byteToString=53, byteTo_boolean=48, byteTo_char=99, byteToByte=120, ByteTo_byte=105}

ByteLegacy2{byteTo_float=49.0, byteTo_double=50.0, byteTo_short=51, byteTo_long=52, byteTo_byte=97, byteTo_int=54, byteToString=5, byteTo_boolean=true, byteTo_char=c, bytToByte=120, copyByteTo_byte=105}

[tm-ms]

I researched in the code and created the test class MainTestStorageLegacyTypeMappingAndroid.
Result:
The problem cannot reproduced on an oracle JVM; even when using the generic MemoryAccessor that is used on Android.

Concsequently, this means there is no bug in the MemoryAccessor implementation.
It seems that android itself must do things kind of differently. But in which way that sometimes only 1 bit flips in a byte to short conversion, remains mysterious.

Maybe it would help to extract the relevant reflection code into a short test class for easy debugability. I'll try to do that.

[tm-ms]

Trying to debug the problem myself via Android Studio with the support from ZJ failed because the main board of my home PC does not support Virtualization.

But it's actually quite trivial to debug if one has a proper system for it:
There is the class BinaryValueTranslators and in it the method #copy_byteTo_float.
That method is called to do the mapping for the field "byteTo_float" in the example.
When I set a breakpoint there on an oracle JVM, it simply gets the value "49" from the sourceAddress and passes the value converted to "49.0" to XMemory#set_float.
That must be the point where things go awry on android.
Can't it get the byte value 49?
Does it convert it incorrectly to float?
Can't it write the float value 49.0 properly?
A simply debugging, just like I did, should give the answer.

[zdenek-jonas]

Nj, the method #copy_byteTo_float. is not called.

[tm-ms]

Ok, then it has to be investigated which translators are used in
AbstractBinaryLegacyTypeHandlerReflective#updateState
or more precisely, which translators are passed in
BinaryLegacyTypeHandlerGenericType#New
Which one and why is passed for the byte-float field.
But that will require a little more debugging and digging.

[zdenek-jonas]

this = {BinaryLegacyTypeHandlerGenericType@14094} 
data = {BinaryLoadItem@14096} "LoadItem OID=1000000000000000028, Type=1000062 one.microstream.legacy.legacy.cross.data.ByteLegacy"
instance = {ByteLegacy2@14097} "ByteLegacy2{byteTo_float=0.0, byteTo_double=0.0, byteTo_short=0, byteTo_long=0, byteTo_byte=0, byteTo_int=0, byteToString=
handler = {BinaryLoader$Default@14095}

[tm-ms]

After a short live debugging on ZJ's Machine per team viewer:
Maybe the order of the primitive fields differ on android and that causes an erroneous behavior by some microstream logic with an ordering oversight.
(see AbstractBinaryLegacyTypeHandlerReflective#updateState regarding used translators and target offsets. Note that for the generic memory accessing, "Memory offsets" are nothing but field indices as that does not really use direct memory accessing but reflection on java.reflect.Fields)
That the reference fields are ordered before the primitive fields is a microstream optimization. The order of the succeeding primitive fields is maintained as provided by the system.

[hg-ms]

One additionaly observed difference between Android and Windows is that
the MultiMatcher.match(....) discards the long, float and double matching

[hg-ms]

This may be a class loader issue.
Under Android the SystemClassLoader used in XReflect.tryResolveType(final String className) does not find the ByteLegacy and ByteLegacy2 classes. This causes the TypeDictionaryParser to set an other TypeDefinitionMemberField type (PersistenceTypeDefinitionMemberFieldGenericSimple instead of PersistenceTypeDefinitionMemberFieldReflective) to be set.
The PersistenceMemberSimilator gets an "null" value for the sourceMember.runtimeQualifier()
resulting in a mismatch. (PersistenceMemberSimilator.calculateSimilarityByName )

Unfortunately i have no idea how to change the SystemClassLoader behavior under Android to prove this ....

[tm-ms]

I removed all references to SystemClassLoader-defaulting type-resolving methods. This time definitely, because I checked all occurances of ClassLoader.getSystemClassLoader() / XReflect#defaultTypeResolvingClassLoader() this time.

PersistenceTypeDictionaryParser now uses the PersistenceTypeResolver which, in turn uses the ClassLoaderProvider instead of the SystemClassLoader.
With that change, the problem should no longer occur.

[zdenek-jonas]

Test successful.