Reading parquet file export from pandas containing np.u32int : PanicException: called Option::unwrap() on a None value

[mhtrinhLIC]

What language are you using?

Python 3.8.0
Polars: 0.13.37
Ubuntu 18.04.5 LTS

The Error

import polars as pl
import pandas as pd 
import numpy as np

N=5
w=pd.DataFrame({'s':np.round(np.random.uniform(1,N,N)).astype(np.uint32)})
out="bug.parquet"
w.to_parquet(out)

print("Reading with pandas")
pd.read_parquet(out)

print("Reading with polars")
pl.read_parquet(out) 

Output :

[...]
    self._df = PyDataFrame.read_parquet(
pyo3_runtime.PanicException: called `Option::unwrap()` on a `None` value

My pip freeze :

argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.0.5
attrs==21.4.0
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.0
cffi==1.15.0
cramjam==2.5.0
cycler==0.11.0
debugpy==1.6.0
decorator==5.1.1
defusedxml==0.7.1
entrypoints==0.4
executing==0.8.3
fastjsonschema==2.15.3
fastparquet==0.8.1
fonttools==4.33.3
fsspec==2022.5.0
importlib-resources==5.7.1
ipykernel==6.13.0
ipython==8.3.0
ipython-genutils==0.2.0
jedi==0.18.1
Jinja2==3.1.2
jsonschema==4.5.1
jupyter-client==7.3.1
jupyter-core==4.10.0
jupyterlab-pygments==0.2.2
kiwisolver==1.4.2
MarkupSafe==2.1.1
matplotlib==3.5.2
matplotlib-inline==0.1.3
mistune==0.8.4
nbclient==0.6.3
nbconvert==6.5.0
nbformat==5.4.0
nest-asyncio==1.5.5
notebook==6.4.11
numpy==1.22.3
packaging==21.3
pandas==1.4.2
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.1.1
pkg_resources==0.0.0
polars==0.13.37
prometheus-client==0.14.1
prompt-toolkit==3.0.29
psutil==5.9.0
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==8.0.0
pycparser==2.21
Pygments==2.12.0
pyparsing==3.0.9
pyrsistent==0.18.1
python-dateutil==2.8.2
pytz==2022.1
pyzmq==23.0.0
scipy==1.8.1
seaborn==0.11.2
Send2Trash==1.8.0
six==1.16.0
soupsieve==2.3.2.post1
stack-data==0.2.0
terminado==0.15.0
tinycss2==1.1.1
tornado==6.1
traitlets==5.2.1.post0
typing_extensions==4.2.0
wcwidth==0.2.5
webencodings==0.5.1
zipp==3.8.0

Do you think it's a pandas issue ?

[jorgecarleitao]

This is certainly a bug in arrow2. Fixed here: jorgecarleitao/arrow2#1090

[ritchie46]

Fixed and merged. Thanks @jorgecarleitao 👍