Wrapping TextIOWrapper around gzip files

[dabeaz]

BPO	10791
Nosy	@jcea, @pitrou, @vstinner, @merwok, @alex, @bitdancer
Files	gzipfile_read1.diff: Add GzipFile.read1() plus tests gzipfile_read1.diff: Fix bug in previous patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-04-04.19:02:53.108>
created_at = <Date 2010-12-29.19:53:51.701>
labels = ['type-feature']
title = 'Wrapping TextIOWrapper around gzip files'
updated_at = <Date 2011-04-07.13:20:42.278>
user = 'https://bugs.python.org/dabeaz'

bugs.python.org fields:

activity = <Date 2011-04-07.13:20:42.278>
actor = 'jcea'
assignee = 'none'
closed = True
closed_date = <Date 2011-04-04.19:02:53.108>
closer = 'pitrou'
components = []
creation = <Date 2010-12-29.19:53:51.701>
creator = 'dabeaz'
dependencies = []
files = ['21531', '21532']
hgrepos = []
issue_num = 10791
keywords = ['patch']
message_count = 30.0
messages = ['124870', '124875', '124876', '124877', '124878', '124879', '124880', '124881', '124882', '124884', '124885', '124886', '124887', '124888', '129187', '129198', '129199', '129200', '129201', '129203', '129205', '129215', '132757', '132899', '132902', '132931', '132948', '132953', '132959', '132960']
nosy_count = 10.0
nosy_names = ['jcea', 'pitrou', 'vstinner', 'nadeem.vawda', 'eric.araujo', 'alex', 'r.david.murray', 'gruszczy', 'dabeaz', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue10791'
versions = ['Python 3.3']

[dabeaz]

Is something like this supposed to work:

>>> import gzip
>>> import io
>>> f = io.TextIOWrapper(gzip.open("foo.gz"),encoding='ascii'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: readable

In a nutshell--reading a .gz file as text.

[bitdancer]

Since GZipFile inherits from BufferedIOBase, and TextIOWrapper is supposed to be designed to wrap a BufferedIOBase object, I would say yes it ought to work. On the other hand there may also be a doc error there, since it may be that TextIOWrapper actually needs to wrap one of the subclasses of BufferedIOBase.

[bitdancer]

Oops. It only has that inheritance in 3.2.

[bitdancer]

Heh, and 2.7. Fixing versions yet again.

[pitrou]

This should be easy to fix, if only the "readable" and "writable" methods are needed. Do you want to try writing a patch?

[dabeaz]

It goes without saying that this also needs to be checked with the bz2 module. A quick check seems to indicate that it has the same problem.

While you're at it, maybe someone could add an 'open' function to bz2 to make it symmetrical with gzip as well :-).

[bitdancer]

bz2 is a pure C module, so that's a very different situation.

[pitrou]

While you're at it, maybe someone could add an 'open' function to bz2
to make it symmetrical with gzip as well :-).

That's a nice idea, but quite orthogonal to this issue.

[dabeaz]

C or not, wrapping a BZ2File instance with a TextIOWrapper to get text still seems like something that someone might want to do. I doubt it would take much modification to give BZ2File instances the required set of methods.

[bitdancer]

Right, but in the bz2 case I think it is a feature request rather than a bugfix. In any case it should be a separate issue.

[pitrou]

C or not, wrapping a BZ2File instance with a TextIOWrapper to get text
still seems like something that someone might want to do. I doubt it
would take much modification to give BZ2File instances the required
set of methods.

BZ2File uses FILE pointers internally so it may be more complicated than
it looks to be (because the methods may not have the right semantics).

[dabeaz]

Do Python devs really view gzip and bz2 as two totally completely different animals? They both have the same functionality and would be used for the same kinds of things. Maybe I'm missing something.

[pitrou]

Do Python devs really view gzip and bz2 as two totally completely
different animals? They both have the same functionality and would be
used for the same kinds of things. Maybe I'm missing something.

Well, the reality of divergent implementation strategies trumps the
theory of API compatibility :) The approach taken by bz2 is IMO
regrettable, but it's not a ten minutes job to write it again from
scratch.

[dabeaz]

Hmmm. Interesting. In the big picture, it might be an interesting project for someone (not necessarily the core devs) to sit down and refactor both of these modules so that they play nice with Python 3 I/O system. Obviously that's a project outside the scope of this bug or the 3.2 release for that matter.