[Python] Add __iter__ method to NativeFile

[asfimport]

Copied from #2114

Currently, NativeFile does not pass the check of pandas for being a "file-like" object because it does not have __iter__ method. It can be easily added using code from native python io module as a reference. Another way would be to subclass io.IOBase.

The exact code of pandas check can be found here.

Code for __iter__ method and io.IOBase can be found here.

Reporter: Wes McKinney / @wesm
Assignee: Dimitri Vorona / @alendit

PRs and other links:

• GitHub Pull Request #2253

_{Note: This issue was originally created as ARROW-2816. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
Subclassing RawIOBase is probably the right thing to do here. IOBase is just a base base class.

[asfimport]

Dimitri Vorona / @alendit:
We can't subclass Python classes in extension types, can we? We could register NativeFile with RawIOBase, though. 

How would you prefer to solve this issue, just add iter + next to NativeFile (should be trivial) or rather extend it to the whole RawIOBase interface and register it, so that it passes other possible checks, too?

[asfimport]

Antoine Pitrou / @pitrou:
NativeFile is a Cython-defined class, it can subclass arbitrary Python classes.

[asfimport]

Dimitri Vorona / @alendit:
Antoine, are you sure? I maybe missunderstand the docs but I think you can only inherit from other extension (i.e. Cython) classes or built-in types. At least I couldn't get it working. The other way around works fine, of course.

[asfimport]

Antoine Pitrou / @pitrou:
Dimitri, hmm, you are right. It could be possible to solve that by using composition: have a cdef class that contains the actual file handle, and then a regular class inheriting from RawIOBase that delegates to methods of the cdef class.

Otherwise we can just implement __iter__ and __next__.

[asfimport]

Dimitri Vorona / @alendit:
Hi Antoine. Since RawIOBase is an ABC, couldn't we just register NativeFile with it, after we've added all the methods? Should take care of any instanceof checks.

[asfimport]

Dimitri Vorona / @alendit:
On a unrelated note: is it normal that I don't receive notification emails if someone posts a comment to an issue I'm watching? The issue creation emails arrive just fine...

 

Disregard that, I think I have it now.

[asfimport]

Antoine Pitrou / @pitrou:
Yes, we should be able to register NativeFile with the ABC.

[asfimport]

Dimitri Vorona / @alendit:
I sent a PR with proposed changes.

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 2253
#2253