Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
FileObject with mode "r" reads bytes instead of string #1441
I am using
With regular Python file object:
>>> fd = open("foo.json", "r") >>> type(fd.read()) <class 'str'>
>>> from gevent.fileobject import FileObject >>> fd = open("/tmp/bla.json", "r") >>> fo = FileObject(fd, mode="r") >>> type(fo.read()) <class 'bytes'>
This makes it impossible to use
The same goes for writing. Opening with mode "w" should expect writing strings but it expects bytes:
>>> from gevent.fileobject import FileObject >>> fd = open("foo.json", "w") >>> fo = FileObject(fd, mode="w") >>> fo.write("bar") Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: a bytes-like object is required, not 'str'
When going over the code (trying to create a PR), I also found some other bugs with this object. The 'U' mode causes some unexpected results in some cases:
from gevent.fileobject import FileObject fd = open("foo", "w") fd.write('bar') # works as expected fo = FileObject(fd, mode="wU") # shouldn't work anyway, but does fo.write('foo') # TypeError: write() argument 1 must be unicode, not str
from gevent.fileobject import FileObject fd = open("foo", "rbU") fo = FileObject(fd, mode="rbU") type(fo.read()) # expected "bytes", got "str"
@jamadden thanks for having a look at this and working out a fix!
Unfortunately there are still a few problems even with the new code. Some of the problems I mentioned earlier in this issue still reproduce, specifically:
Should I open a new issue for these problems?
In addition, the new behavior mentioned: "If 't' is given, they will always work with unicode strings" sort of breaks compatibility with regular file objects on Python 2. Opening a file in Python 2 with "t" reads "str" type, but now gevent FileObject will read "unicode" type, which, like the original problem I encountered, can cause trouble when passing a
That appears to be a legit issue. It's obviously untested. I'll look into it. Thanks!
And it is, raising a
It depends on how the "regular file object" was opened.
I think there are no places that