Skip to content

[query] Improve error message from hadoop_ls when file does not exist#10007

Merged
danking merged 1 commit intohail-is:mainfrom
nawatts:hadoop-ls-file-does-not-exist
Feb 8, 2021
Merged

[query] Improve error message from hadoop_ls when file does not exist#10007
danking merged 1 commit intohail-is:mainfrom
nawatts:hadoop-ls-file-does-not-exist

Conversation

@nawatts
Copy link
Contributor

@nawatts nawatts commented Feb 8, 2021

As described in #9600, hl.hadoop_ls currently raises a NullPointerException when called with a path that does not exist. The error originates in is.hail.io.fs.HadoopFS.listStatus.

The hadoop.fs.FileSystem docs list two forms of globStatus:

  • public FileStatus[] globStatus(Path pathPattern)
  • public FileStatus[] globStatus(Path pathPattern, PathFilter filter)

The first makes no mention of returning null. The second however, says:

Returns: null if pathPattern has no glob and the path does not exist an empty array if pathPattern has a glob and no path matches it else an array of FileStatus objects matching the pattern

This matches the behavior seen with hl.hadoop_ls.

This change checks for a null value returned from globStatus and raises a FileNotFoundException in that case.

Resolves #9600

@nawatts
Copy link
Contributor Author

nawatts commented Feb 8, 2021

Scorecard returned @johnc1231

Copy link
Contributor

@johnc1231 johnc1231 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great fix, thank you!

@danking danking merged commit 8d4d1d8 into hail-is:main Feb 8, 2021
@nawatts nawatts deleted the hadoop-ls-file-does-not-exist branch May 6, 2021 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

hadoop_ls NullPointerException when file doesn't exist

3 participants