XLANG serialisation fails python -> java

[goldv]

Search before asking

• I had searched in the issues and found no similar issues.

Version

python:
pyfury == 0.10.2

java:
implementation group: 'org.apache.fury', name: 'fury-core', version: '0.10.1'

Component(s)

Java, Python

Minimal reproduce step

from dataclasses import dataclass

import pyfury


@dataclass
class SomeClass1:
    f1: int = None
    f2: int = None


if __name__ == "__main__":
    f = pyfury.Fury(language=pyfury.Language.XLANG, ref_tracking=False, require_class_registration=True)

    f.register_type(SomeClass1, typename="example.SomeClass1")

    obj = SomeClass1(f1=1, f2=2)

    data = f.serialize(obj)

    print("bytes:", list(data))

    obj2 = f.deserialize(data)
    print("Deserialized:", obj2)

    # Save to file for Java to read
    with open("/data/class.bin", "wb") as file:
        file.write(data)

public static class SomeClass1 {
        public int f1;
        public int f2;
}

Fury fury = Fury.builder().withLanguage(Language.XLANG)
                .withRefTracking(false)
                .requireClassRegistration(true)
                .build();

fury.register(SomeClass1.class, "example.SomeClass1");

SomeClass1 o = new SomeClass1();
o.f1 = 1;
o.f2 = 2;

// Read bytes from file
byte[] bytes = Files.readAllBytes(Paths.get("/data/class.bin"));

System.out.println("bytes: " + Arrays.toString(unsigned(bytes)));

// Deserialize
SomeClass1 obj = (SomeClass1) fury.deserialize(bytes);

What did you expect to see?

The object should be serialised in python and deserialized in java

What did you see instead?

Serialising to a file from python and then attempting to deserialize in java gives the following:

Exception in thread "main" java.lang.NullPointerException
at org.apache.fury.util.Preconditions.checkNotNull(Preconditions.java:26)
at org.apache.fury.Fury.xreadNonRef(Fury.java:1058)
at org.apache.fury.Fury.xreadRef(Fury.java:1000)
at org.apache.fury.Fury.xdeserializeInternal(Fury.java:863)
at org.apache.fury.Fury.deserialize(Fury.java:801)
at org.apache.fury.Fury.deserialize(Fury.java:727)
at qtify.event.codec.ReferenceExample.main(ReferenceExample.java:41)

the actual serialised data is:

python: [212, 98, 6, 2, 255, 17, 10, 1, 18, 224, 99, 214, 64, 16, 2, 88, 113, 130, 56, 88, 9, 37, 168, 145, 64, 0, 0, 255, 2, 255, 4]
java: [212, 98, 6, 1, 60, 0, 0, 0, 0, 0, 0, 0, 255, 0, 1, 36, 0, 127, 128, 47, 225, 165, 129, 250, 101, 120, 97, 109, 112, 108, 101, 46, 83, 111, 109, 101, 67, 108, 97, 115, 115, 49, 177, 64, 0, 0, 255, 7, 0, 1, 0, 0, 0, 255, 7, 0, 2, 0, 0, 0]

Anything Else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[chaokunyang]

Hi @goldv , you need to Integer for int type in java, currently we don't support Optional in python, we take all fields as nullable. This will be fixed after we support Optional for python field. When this is supported, we can take f1: int as not null, and take f1: Optional[int] as nullable

[goldv]

hi @chaokunyang , thanks for the response. I've tried your suggestion but still seeing the same error.

[chaokunyang]

@dataclass
class SomeClass1:
    f1: pyfury.Int32Type = None
    f2: pyfury.Int32Type = None

public static class SomeClass1 {
    public Integer f1;
    public Integer f2;
}

Could you try with above classes?

[goldv]

still seeing the same error with that change.

[chaokunyang]

It should work now:

@dataclass
class SomeClass1:
    f1: pyfury.int32 = None
    f2: pyfury.int32 = None

public static class SomeClass1 {
    public int f1;
    public int f2;
}