Data Reader

[zyhbili]

I encounter some bugs when read the preprocessed data.
Traceback (most recent call last):
File "test.py", line 14, in
value = pa.deserialize(buf)
File "pyarrow/serialization.pxi", line 550, in pyarrow.lib.deserialize
File "pyarrow/serialization.pxi", line 555, in pyarrow.lib._deserialize
File "pyarrow/serialization.pxi", line 461, in pyarrow.lib._read_serialized
File "pyarrow/error.pxi", line 115, in pyarrow.lib.check_status
OSError: Expected IPC message of type sparse tensor but got tensor

Here is my code. My pyarrow==8.0.0. Could you provide the data reader or some instructions?

txn = env.begin()  
key = '%010d'%0  
buf=txn.get(key.encode('ascii'))  
value = pyarrow.deserialize(buf)

[alvinliu0]

Hi :)

Sorry for the late reply, the version of pyarrow we used is 0.14.1. I test and find that the version 8.0.0 is incompatible with this environment. You could install the package via pip install pyarrow==0.14.1

Hope this could fix your problem!