ARROW-2600: [Python] Add additional LocalFileSystem filesystem methods #2060
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xhochy
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When starting to dig into ticket 1360 I noticed some of the methods could be added for the LocalFileSystem. I wanted to get this one reviewed first, but also ask:
I have the hdfs docker image linked in ticket 1360. What I'm not sure of is how pytest and the current CI will spin up an HDFS image, and should I create files on HDFS for something like the cat method to use similar to how I did in this PR or is there a different/better way to handle test for this?
For 1319 do we know how much can be implemented in Python? I see a couple tickets tagged C++/Python so I didn't know if once I got into this ticket if there are certain pieces that are known to require C++ interaction or if should all be available to do in Python?
Lastly I styled this off cat in Ubuntu 16.04 (I doubt there is much if any difference across versions and flavors), but I was wondering how we want to implement the methods if we are worried about extra flags the terminal supports, things like printing the file name if it's a list of files and not printing with an individual file or just implementing the core functionality of printing the file lines? That's just one example, but I figured I would check as I move on to the other filesystem commands.