readline() method

[coreyhuinker]

This would be useful for extracting the header from a csv.

[piskvorky]

I agree. Can you implement it?

[coreyatmoat]

I'll give it a shot.

[piskvorky]

@coreyatmoat what's your progress?

[coreyatmoat]

Sadly no. A while back a co-worker wrote https://gist.github.com/coreyatmoat/c961f99dfb0cfabdda54 as a sort of work around, but I haven't had time to figure out how it'd fit into your framework.

[Mgutjahr]

+1

[mpenkov]

I'm having a look at this now, since one of my projects needs this to work too :)

Is there any reason why we need to maintain independent pointers for read and __iter__? The default implementation of readline shares the file pointer with read (and other I/O operations), to the best of my knowledge.

[piskvorky]

Thanks @mpenkov .

The reason for the "independent pointer" was related to difficulties with buffering IIRC, but was only technical (CC @ziky90 ). That behaviour is not a part of the API contract (I don't think it's even documented).

If we can make a single pointer work, that's fine, even preferable.

[mpenkov]

@piskvorky OK. I think I've got it working. Have a look here: https://github.com/mpenkov/smart_open/commit/aca3a18358afb5f42b5f7e5c1355e8ff93a9bcea

I made a separate branch (readline) for it, since an open pull request (#75) is currently using the clone's master. This branch also contains the gzip changes.

What do you think is the cleanest way of getting this into your repo? The simplest I can think of:

1. Close Resolve issue #12 (unable to read gz from S3) #75
2. Open a new pull request from my new readline branch

Another way is:

1. You guys merge Resolve issue #12 (unable to read gz from S3) #75 whenever you're ready
2. I update my clone
3. I make a new pull request for the readline stuff

Let me know if there's a simpler or more convenient way.

[piskvorky]

Great stuff! @tmylk please review.

Re. branches: it's your call. If splitting the changes into two separate PRs is too complicated, doing both in one branch is fine with me. We appreciate your work, this is some much needed functionality.

[mpenkov]

@piskvorky @tmylk I've authored a request to pull from my readline to your master (#76). Let me know if you need anything else from me.

[mpenkov]

@piskvorky @tmylk I think we can close this now. 78c461e resolved this.