This repository was archived by the owner on May 9, 2024. It is now read-only.
This repository was archived by the owner on May 9, 2024. It is now read-only.
SIGSEGV on importing arrow table with null columns #393
Description
Code to reproduce:
import modin.pandas as pd
df = pd.DataFrame(columns=["a"])
print(df + df)Output:
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f65ab0f32f5, pid=273311, tid=273311
#
# JRE version: OpenJDK Runtime Environment (17.0.3) (build 17.0.3-internal+0-adhoc..src)
# Java VM: OpenJDK 64-Bit Server VM (17.0.3-internal+0-adhoc..src, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# C [libArrowStorage.so+0x2a2f5] ArrowStorage::computeStats(std::shared_ptr<arrow::ChunkedArray>, hdk::ir::Type const*, ChunkStats&)+0x85
The error occurs when importing an arrow table with null columns:
import pyarrow as pa
from modin.experimental.core.execution.native.implementations.hdk_on_native.hdk_worker import (
HdkWorker,
)
table = pa.table([[]], schema=pa.schema([pa.field("A", pa.null())]))
HdkWorker().import_arrow_table(table, "test")I'm not sure if adding support for null columns is required, but, anyway, it shouldn't crash and an exception should be thrown instead.
Metadata
Metadata
Assignees
Labels
No labels