Skip to content

Conversation

@IanButterworth
Copy link
Member

@IanButterworth IanButterworth commented Jul 13, 2020

This adds more eager checking for path errors during load & save for clearer error messages.
Fixes #263


Loading an a nonexistant file

Before:

julia> FileIO.load("fake/fake.png")
Errors encountered while loading "fake/fake.png".
All errors:
===========================================
Failed to open fake/fake.png
===========================================
ArgumentError: Package QuartzImageIO not found in current path:
- Run `import Pkg; Pkg.add("QuartzImageIO")` to install the QuartzImageIO package.

===========================================
ArgumentError: Package ImageMagick not found in current path:
- Run `import Pkg; Pkg.add("ImageMagick")` to install the ImageMagick package.

===========================================

Fatal error:
ERROR: Failed to open fake/fake.png
...

This PR:

julia> FileIO.load("fake/fake.png")
ERROR: ArgumentError: No file exists at given path: fake/fake.png
Stacktrace:
...

Trying to save into a directory that doesn't exist

Before:

julia> FileIO.save("fake_dir/image.png", rand(UInt8,10,10))
Errors encountered while saving "fake_dir/image.png".
All errors:
===========================================
Could not open fake_dir/image.png for writing
===========================================
ArgumentError: Package QuartzImageIO not found in current path:
- Run `import Pkg; Pkg.add("QuartzImageIO")` to install the QuartzImageIO package.

===========================================
ArgumentError: Package ImageMagick not found in current path:
- Run `import Pkg; Pkg.add("ImageMagick")` to install the ImageMagick package.

===========================================
ArgumentError: Argument does not support conversion to png.
===========================================

Fatal error:
ERROR: Could not open fake_dir/image.png for writing
Stacktrace:
...

This PR:

julia> FileIO.save("fake_dir/image.png", rand(UInt8,10,10))
ERROR: ArgumentError: Directory of file path does not exist: fake_dir/image.png
Stacktrace:
...

UPDATE: Now mkpath() is run on the parent directory, if it doesn't exist and no error is thrown


Trying to save to a path that's a directory

Before

julia> FileIO.save(format"PNG", "existing_dir", rand(UInt8,10,10))
Errors encountered while saving "existing_dir".
All errors:
===========================================
Could not open existing_dir for writing
===========================================
ArgumentError: Package QuartzImageIO not found in current path:
- Run `import Pkg; Pkg.add("QuartzImageIO")` to install the QuartzImageIO package.

===========================================
ArgumentError: Package ImageMagick not found in current path:
- Run `import Pkg; Pkg.add("ImageMagick")` to install the ImageMagick package.

===========================================
ArgumentError: Argument does not support conversion to png.
===========================================

Fatal error:
ERROR: Could not open existing_dir for writing
Stacktrace:
...

This PR:

julia> FileIO.save(format"PNG", "existing_dir", rand(UInt8,10,10))
ERROR: ArgumentError: Given file path is a directory: existing_dir
Stacktrace:
...

cc. @johnnychen94 @timholy

@IanButterworth
Copy link
Member Author

By the way, is ArgumentError the best option to throw here?

@codecov
Copy link

codecov bot commented Jul 13, 2020

Codecov Report

Merging #265 into master will increase coverage by 0.28%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #265      +/-   ##
==========================================
+ Coverage   76.02%   76.31%   +0.28%     
==========================================
  Files           8        8              
  Lines         413      418       +5     
==========================================
+ Hits          314      319       +5     
  Misses         99       99              
Impacted Files Coverage Δ
src/loadsave.jl 86.84% <100.00%> (+0.92%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 882dc75...f50c2b5. Read the comment docs.

Copy link
Member

@johnnychen94 johnnychen94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ianshmean Thanks! I've recently seen a lot of people bit by this obscure error message, even for core devs in JuliaImages. For example, JuliaImages/Images.jl#888 JuliaImages/ImageEdgeDetection.jl#7

One reason I didn't put my hands on this is that I feel it over-eager to check whether the directory exists when saving images in FileIO; I feel that should depend on backends implementations. The other two checks look good to me.

src/loadsave.jl Outdated
unknown(q) && throw(UnknownFormat(q))
if q isa File
isdir(filename(q)) && throw(ArgumentError("Given file path is a directory: $(filename(q))"))
!isdir(dirname(filename(q))) && throw(ArgumentError("Directory of file path does not exist: $(filename(q))"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My best guess is that some backends might support path creation before writing, so checking if the directory exists is perhaps too eagerly for a general IO frontend.

Checking whether the file path is a directory looks good to me.

@IanButterworth
Copy link
Member Author

Perhaps a solution could be to always mkpath() before passing to the backends? It's been mentioned before that that would be nice default behavior.

@johnnychen94
Copy link
Member

Not sure how others think, mkpath by default sounds a good solution to me

@Octogonapus
Copy link

Perhaps a solution could be to always mkpath() before passing to the backends? It's been mentioned before that that would be nice default behavior.

I would also like this to be the default.

@IanButterworth
Copy link
Member Author

Ok, great. I've made that the case now

Copy link
Member

@johnnychen94 johnnychen94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IanButterworth
Copy link
Member Author

Should we wait for more reviews?

@timholy
Copy link
Member

timholy commented Jul 17, 2020

I think it's possible we'll be troubled by the path-creation, but this is undeniably better and might be perfect. Let's try it and see how we like it.

I think this merits a minor version bump, though? As a new feature? (That won't be breaking as we're over 1.0.)

@timholy timholy merged commit c106b1f into master Jul 17, 2020
@timholy timholy deleted the ib/existence_checks branch July 17, 2020 20:36
@IanButterworth
Copy link
Member Author

Minor bump sounds right. Thanks!

@timholy
Copy link
Member

timholy commented Jul 17, 2020

Thanks for fixing this, BTW!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

load() doesn't check whether file exists before offering alternative IO packages

5 participants