zlib failure in PyArrow 0.11.0

[amachanic]

The following produces an exception on PyArrow 0.11.0: "zlib deflate failed, output buffer too small"

It works fine on 0.10.0, so I assume this is a regression?

import io
import pyarrow
from pyarrow import parquet

tbl = pyarrow.Table.from_arrays([pyarrow.array(['abc', 'def'])], ['some_col'])

f = io.BytesIO()
parquet.write_table(tbl, f, compression='gzip')

[kszucs]

Although I can't reproduce it, the error must come from here.

How did You install pyarrow? Which platform?
Could You please create a JIRA ticket instead?

[wesm]

I think we are dynamically linking to zlib now in 0.11 so @amachanic could you let us know what other versions of zlib do you have installed on your system?

[amachanic]

@kszucs I installed pyarrow via pip. I had initially tested this on Amazon Linux (which I guess is Red Hat). Just tested on a couple of other machines: CentOS 7, and Ubuntu 16.04, and the repro works (which is to say, throws an exception) on both. I also just tested on Windows 7, and the repro does NOT throw an exception there.

Happy to create a JIRA ticket.

@wesm: both the Amazon Linux and CentOS machines have zlib 1.2.7-17. On Ubuntu, looks like it's 1.2.8 -- but full disclosure, I'm not sure I checked properly there. And I'm not sure at all how to check on Windows.

[amachanic]

JIRA ticket here. Thank you for your attention to this issue.

[tsellers-r7]

It looks like the fix landed 2 days ago. Is there a projected date for when the code will be released and show up on PyPi?

Thanks.

[wesm]

We need to release 0.11.1. I am going to work on that today; so reckon updated packages would be on PyPI within the next 96 hours or so

[amachanic]

@wesm many thanks!

[xhochy]

The fix was uploaded on PyPI some days ago.