Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] Segmentation fault when writing empty ListType column to Parquet #18976

Closed
asfimport opened this issue May 16, 2018 · 3 comments
Closed

Comments

@asfimport
Copy link
Collaborator

Context Is the following: I am currently dealing with sparse column serialization in parquet. In some cases, many lines are empty I can also have columns containing only empty lists.
However I got a segmentation fault when I try to write in parquet thoses columns filled only with empty lists.

Here is a simple code snipet reproduces the segmentation fault I had:



In [1]: import pyarrow as pa

In [2]: import pyarrow.parquet as pq

In [3]: pa_ar = pa.array([[],[]],pa.list_(pa.int32()))

In [4]: table = pa.Table.from_arrays([pa_ar],["test"])

In [5]: pq.write_table(
   ...:     table=table,
   ...:     where="test.parquet",
   ...:     compression="snappy",
   ...:     flavor="spark"
   ...: )
Segmentation fault

May I have it fixed?

Best

Jacques

Reporter: jacques
Assignee: Krisztian Szucs / @kszucs

PRs and other links:

Note: This issue was originally created as ARROW-2591. Please see the migration documentation for further details.

@asfimport
Copy link
Collaborator Author

Wes McKinney / @wesm:

May I have it fixed?

[~jafournier] for future reference, it isn't ideal in open source projects to ask volunteers to fix bugs for you in this way. After you report the bug; if it is deemed a priority by another developer, they may fix it. Otherwise, if they do not fix it, and you need the fix sooner, we would be glad to accept a pull request.

@asfimport
Copy link
Collaborator Author

Krisztian Szucs / @kszucs:
Issue resolved by pull request 3028
#3028

@asfimport
Copy link
Collaborator Author

jacques:
many thanks!

@asfimport asfimport added this to the 0.12.0 milestone Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants