ARROW-2919: [C++/Python] Improve HdfsFile error messages, fix Python unit test suite #3209
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Change-Id: If58d642c2d627bcf9d5902064d03d13f8b5bd4fe
…ue to non-existent file or bad port Change-Id: Ic490a94a3837a609066c29a815efd1902a2b7bf3
pitrou
reviewed
Dec 18, 2018
| auto path = this->ScratchPath("path-does-not-exist"); | ||
|
|
||
| HdfsPathInfo info; | ||
| Status s = this->client_->GetPathInfo(path, &info); |
There was a problem hiding this comment.
Assert that the Status is a particular error?
python/pyarrow/parquet.py
Outdated
|
|
||
| self.fs, _ = _get_filesystem_and_path(filesystem, a_path) | ||
| self.paths = path_or_paths |
There was a problem hiding this comment.
This temporary assignment looks a bit superfluous. You could just access path_or_paths below.
Change-Id: Ib7b69f3287876b95db5b2b03d3e9c3daf5f9c26d
Change-Id: I35dec2a1a19cdaf1688281b6798680e79bf95501
cav71
pushed a commit
to cav71/arrow
that referenced
this pull request
Dec 22, 2018
…unit test suite This also resolves ARROW-3957 and ARROW-4053. Summary: * Properly initialize NativeFile when opening from HDFS. This was broken when the "closed" property was added and some other refactoring, and wasn't caught because these tests aren't being run regularly * Slightly improves the handling of filesystem URIs -- there were some tests that failed without these changes because the docker-compose HDFS containers don't allow writes from $USER * Improve error message when calling "info" on a file that does not exist * Improve error message when calling `ls` on a directory that does not exist * Suggest checking whether you are connecting to the right HDFS port when getting errno 255 Author: Wes McKinney <wesm+git@apache.org> Closes apache#3209 from wesm/ARROW-2919 and squashes the following commits: b11e5b6 <Wes McKinney> Restore arrow_dependencies to Gandiva dependencies 20e8784 <Wes McKinney> Code review comments 4ba93bb <Wes McKinney> More helpful error messages when GetPathInfo or ListDirectory fails due to non-existent file or bad port 3c67ea6 <Wes McKinney> Basic fixes to get Python unit tests passing again
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This also resolves ARROW-3957 and ARROW-4053.
Summary:
lson a directory that does not exist