You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context Is the following: I am currently dealing with sparse column serialization in parquet. In some cases, many lines are empty I can also have columns containing only empty lists.
However I got a segmentation fault when I try to write in parquet thoses columns filled only with empty lists.
Here is a simple code snipet reproduces the segmentation fault I had:
In [1]: import pyarrow as pa
In [2]: import pyarrow.parquet as pq
In [3]: pa_ar = pa.array([[],[]],pa.list_(pa.int32()))
In [4]: table = pa.Table.from_arrays([pa_ar],["test"])
In [5]: pq.write_table(
...: table=table,
...: where="test.parquet",
...: compression="snappy",
...: flavor="spark"
...: )
Segmentation fault
[~jafournier] for future reference, it isn't ideal in open source projects to ask volunteers to fix bugs for you in this way. After you report the bug; if it is deemed a priority by another developer, they may fix it. Otherwise, if they do not fix it, and you need the fix sooner, we would be glad to accept a pull request.
Context Is the following: I am currently dealing with sparse column serialization in parquet. In some cases, many lines are empty I can also have columns containing only empty lists.
However I got a segmentation fault when I try to write in parquet thoses columns filled only with empty lists.
Here is a simple code snipet reproduces the segmentation fault I had:
May I have it fixed?
Best
Jacques
Reporter: jacques
Assignee: Krisztian Szucs / @kszucs
PRs and other links:
Note: This issue was originally created as ARROW-2591. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: