Skip to content

[Parquet][Python] Crash when accessing Table.schema.metadata after to_pandas(self_destruct=True) #50698

Description

@Alvaro-Kothe

Describe the bug, including details regarding any error messages, version, and platform.

Accessing the metadata of a PyArrow Table after converting it to a pandas with self_destruct=True causes a segmentation fault.

This was originally reported in pandas-dev/pandas#66509 and it has been adapted to a pure pyarrow path.

import pyarrow as pa
import pyarrow.parquet as pq
import io

table = pa.table({"0": [0.25]})
with io.BytesIO() as buf:
    pq.write_table(table, buf)

    pa_table = pq.read_table(
        buf,
        columns=None,
        filesystem=None,
        filters=None,
        use_pandas_metadata=True,
    )

df = pa_table.to_pandas(self_destruct=True)
this_will_crash = pa_table.schema.metadata

I ran it through Python built with ASan, here is the stack trace it provided:

==506442==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x7bf21631cc0a bp 0x7ffde3402dc0 sp 0x7ffde3402da0 T0)
==506442==The signal is caused by a READ memory access.
==506442==Hint: address points to the zero page.
    #0 0x7bf21631cc0a in __pyx_f_7pyarrow_3lib_6Schema_init_schema(__pyx_obj_7pyarrow_3lib_Schema*, std::shared_ptr<arrow::Schema> const&) (.lto_priv.0) (/home/alvaro/projects/oss/pandas/.venv-asan/lib/python3.14/site-packages/pyarrow/lib.cpython-314-x86_64-linux-gnu.so+0x11cc0a) (BuildId: db7bac4d3a03ff0fd1f0d2214fc9112eccb77656)
    #1 0x7bf21648a1b0 in pyarrow_wrap_schema(std::shared_ptr<arrow::Schema> const&) (/home/alvaro/projects/oss/pandas/.venv-asan/lib/python3.14/site-packages/pyarrow/lib.cpython-314-x86_64-linux-gnu.so+0x28a1b0) (BuildId: db7bac4d3a03ff0fd1f0d2214fc9112eccb77656)
    #2 0x7bf2164ab0c9 in __pyx_getprop_7pyarrow_3lib_5Table_schema(_object*, void*) (.lto_priv.0) (/home/alvaro/projects/oss/pandas/.venv-asan/lib/python3.14/site-packages/pyarrow/lib.cpython-314-x86_64-linux-gnu.so+0x2ab0c9) (BuildId: db7bac4d3a03ff0fd1f0d2214fc9112eccb77656)
    #3 0x0000008f007a in _PyObject_GenericGetAttrWithDict /home/alvaro/opt/Python-3.14.6/Objects/object.c:1848:19
    #4 0x0000008ed602 in PyObject_GetAttr /home/alvaro/opt/Python-3.14.6/Objects/object.c
    #5 0x000000ce2aff in _PyEval_EvalFrameDefault /home/alvaro/opt/Python-3.14.6/Python/generated_cases.c.h:8027:40
    #6 0x000000caec33 in _PyEval_Vector /home/alvaro/opt/Python-3.14.6/Python/ceval.c:2110:12
    #7 0x000000caec33 in PyEval_EvalCode /home/alvaro/opt/Python-3.14.6/Python/ceval.c:982:21
    #8 0x000000f6a7a7 in run_mod /home/alvaro/opt/Python-3.14.6/Python/pythonrun.c:1460:19
    #9 0x000000f646d5 in pyrun_file /home/alvaro/opt/Python-3.14.6/Python/pythonrun.c:1294:15
    #10 0x000000f61f1f in _PyRun_SimpleFileObject /home/alvaro/opt/Python-3.14.6/Python/pythonrun.c:521:13
    #11 0x000000f61214 in _PyRun_AnyFileObject /home/alvaro/opt/Python-3.14.6/Python/pythonrun.c:81:15
    #12 0x0000010312a6 in pymain_run_file_obj /home/alvaro/opt/Python-3.14.6/Modules/main.c:410:15
    #13 0x0000010312a6 in pymain_run_file /home/alvaro/opt/Python-3.14.6/Modules/main.c:429:15
    #14 0x00000102e965 in pymain_run_python /home/alvaro/opt/Python-3.14.6/Modules/main.c:695:21
    #15 0x00000102e965 in Py_RunMain /home/alvaro/opt/Python-3.14.6/Modules/main.c:776:5
    #16 0x0000010300c6 in pymain_main /home/alvaro/opt/Python-3.14.6/Modules/main.c:806:12
    #17 0x000001030344 in Py_BytesMain /home/alvaro/opt/Python-3.14.6/Modules/main.c:830:12
    #18 0x7ff227a71680 in __libc_start_call_main (/lib64/libc.so.6+0x3680) (BuildId: 17f2e1fd905f485786f6fd6e3bede4ad737137e7)
    #19 0x7ff227a71797 in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x3797) (BuildId: 17f2e1fd905f485786f6fd6e3bede4ad737137e7)
    #20 0x0000004031e4 in _start (/home/alvaro/opt/python-asan/bin/python3.14+0x4031e4) (BuildId: a933e09ec4da057744c3a360de3bb0f576714241)

==506442==Register values:
rax = 0x00007bf2165c9250  rbx = 0x00007c5228d2cba0  rcx = 0x00000f7e42cb72b9  rdx = 0x0000000000000028  
rdi = 0x00007c5228d2cba0  rsi = 0x0000000000000008  rbp = 0x00007ffde3402dc0  rsp = 0x00007ffde3402da0  
 r8 = 0x00000f8a451a5976   r9 = 0x0000000000000000  r10 = 0x00000f8a451a597a  r11 = 0x0000000000000000  
r12 = 0x0000000000000008  r13 = 0x00007bf225aa8180  r14 = 0x00000f8a451a3d9d  r15 = 0x00007c3226f64458  

Component(s)

Parquet, Python

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions