[Python/C++] Conversion of dict encoded null column fails in parquet writing when using RowGroups

[asfimport]

Conversion of dict encoded null column fails in parquet writing when using RowGroups

import pyarrow.parquet as pq
import pandas as pd
import pyarrow as pa
df = pd.DataFrame({"col": [None] * 100, "int": [1.0] * 100})
df = df.astype({"col": "category"})
table = pa.Table.from_pandas(df)
buf = pa.BufferOutputStream()
pq.write_table(
    table,
    buf,
    version="2.0",
    chunk_size=10,
)

fails with

pyarrow.lib.ArrowIOError: Column 2 had 100 while previous column had 10

Reporter: Florian Jetter / @fjetter
Assignee: Wes McKinney / @wesm

PRs and other links:

• GitHub Pull Request #5107

_{Note: This issue was originally created as ARROW-5085. Please see the migration documentation for further details.}

[asfimport]

Wes McKinney / @wesm:
This is now fixed in master, presumably by ARROW-3246. I'll add a unit test. Note that version=2.0 files should not be used, see PARQUET-458

[asfimport]

Wes McKinney / @wesm:
Hm, reading the file back causes a segfault. Taking a look

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 5107
#5107