Describe the bug, including details regarding any error messages, version, and platform.
Accessing the metadata of a PyArrow Table after converting it to a pandas with self_destruct=True causes a segmentation fault.
This was originally reported in pandas-dev/pandas#66509 and it has been adapted to a pure pyarrow path.
import pyarrow as pa
import pyarrow.parquet as pq
import io
table = pa.table({"0": [0.25]})
with io.BytesIO() as buf:
pq.write_table(table, buf)
pa_table = pq.read_table(
buf,
columns=None,
filesystem=None,
filters=None,
use_pandas_metadata=True,
)
df = pa_table.to_pandas(self_destruct=True)
this_will_crash = pa_table.schema.metadata
I ran it through Python built with ASan, here is the stack trace it provided:
==506442==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x7bf21631cc0a bp 0x7ffde3402dc0 sp 0x7ffde3402da0 T0)
==506442==The signal is caused by a READ memory access.
==506442==Hint: address points to the zero page.
#0 0x7bf21631cc0a in __pyx_f_7pyarrow_3lib_6Schema_init_schema(__pyx_obj_7pyarrow_3lib_Schema*, std::shared_ptr<arrow::Schema> const&) (.lto_priv.0) (/home/alvaro/projects/oss/pandas/.venv-asan/lib/python3.14/site-packages/pyarrow/lib.cpython-314-x86_64-linux-gnu.so+0x11cc0a) (BuildId: db7bac4d3a03ff0fd1f0d2214fc9112eccb77656)
#1 0x7bf21648a1b0 in pyarrow_wrap_schema(std::shared_ptr<arrow::Schema> const&) (/home/alvaro/projects/oss/pandas/.venv-asan/lib/python3.14/site-packages/pyarrow/lib.cpython-314-x86_64-linux-gnu.so+0x28a1b0) (BuildId: db7bac4d3a03ff0fd1f0d2214fc9112eccb77656)
#2 0x7bf2164ab0c9 in __pyx_getprop_7pyarrow_3lib_5Table_schema(_object*, void*) (.lto_priv.0) (/home/alvaro/projects/oss/pandas/.venv-asan/lib/python3.14/site-packages/pyarrow/lib.cpython-314-x86_64-linux-gnu.so+0x2ab0c9) (BuildId: db7bac4d3a03ff0fd1f0d2214fc9112eccb77656)
#3 0x0000008f007a in _PyObject_GenericGetAttrWithDict /home/alvaro/opt/Python-3.14.6/Objects/object.c:1848:19
#4 0x0000008ed602 in PyObject_GetAttr /home/alvaro/opt/Python-3.14.6/Objects/object.c
#5 0x000000ce2aff in _PyEval_EvalFrameDefault /home/alvaro/opt/Python-3.14.6/Python/generated_cases.c.h:8027:40
#6 0x000000caec33 in _PyEval_Vector /home/alvaro/opt/Python-3.14.6/Python/ceval.c:2110:12
#7 0x000000caec33 in PyEval_EvalCode /home/alvaro/opt/Python-3.14.6/Python/ceval.c:982:21
#8 0x000000f6a7a7 in run_mod /home/alvaro/opt/Python-3.14.6/Python/pythonrun.c:1460:19
#9 0x000000f646d5 in pyrun_file /home/alvaro/opt/Python-3.14.6/Python/pythonrun.c:1294:15
#10 0x000000f61f1f in _PyRun_SimpleFileObject /home/alvaro/opt/Python-3.14.6/Python/pythonrun.c:521:13
#11 0x000000f61214 in _PyRun_AnyFileObject /home/alvaro/opt/Python-3.14.6/Python/pythonrun.c:81:15
#12 0x0000010312a6 in pymain_run_file_obj /home/alvaro/opt/Python-3.14.6/Modules/main.c:410:15
#13 0x0000010312a6 in pymain_run_file /home/alvaro/opt/Python-3.14.6/Modules/main.c:429:15
#14 0x00000102e965 in pymain_run_python /home/alvaro/opt/Python-3.14.6/Modules/main.c:695:21
#15 0x00000102e965 in Py_RunMain /home/alvaro/opt/Python-3.14.6/Modules/main.c:776:5
#16 0x0000010300c6 in pymain_main /home/alvaro/opt/Python-3.14.6/Modules/main.c:806:12
#17 0x000001030344 in Py_BytesMain /home/alvaro/opt/Python-3.14.6/Modules/main.c:830:12
#18 0x7ff227a71680 in __libc_start_call_main (/lib64/libc.so.6+0x3680) (BuildId: 17f2e1fd905f485786f6fd6e3bede4ad737137e7)
#19 0x7ff227a71797 in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x3797) (BuildId: 17f2e1fd905f485786f6fd6e3bede4ad737137e7)
#20 0x0000004031e4 in _start (/home/alvaro/opt/python-asan/bin/python3.14+0x4031e4) (BuildId: a933e09ec4da057744c3a360de3bb0f576714241)
==506442==Register values:
rax = 0x00007bf2165c9250 rbx = 0x00007c5228d2cba0 rcx = 0x00000f7e42cb72b9 rdx = 0x0000000000000028
rdi = 0x00007c5228d2cba0 rsi = 0x0000000000000008 rbp = 0x00007ffde3402dc0 rsp = 0x00007ffde3402da0
r8 = 0x00000f8a451a5976 r9 = 0x0000000000000000 r10 = 0x00000f8a451a597a r11 = 0x0000000000000000
r12 = 0x0000000000000008 r13 = 0x00007bf225aa8180 r14 = 0x00000f8a451a3d9d r15 = 0x00007c3226f64458
Component(s)
Parquet, Python
Describe the bug, including details regarding any error messages, version, and platform.
Accessing the metadata of a PyArrow
Tableafter converting it to a pandas withself_destruct=Truecauses a segmentation fault.This was originally reported in pandas-dev/pandas#66509 and it has been adapted to a pure pyarrow path.
I ran it through Python built with ASan, here is the stack trace it provided:
Component(s)
Parquet, Python