Disallow reading directories as files

[jakobnissen]

Right now, you can do:

julia> open("/Users/jakobnissen/Documents/") do file
           read(file)
       end
0-element Array{UInt8,1}

That doesn't make much sense to me - presumably, attempting to read a directory is always a mistake, so we may as well error there. In fact, I discovered this because I accidentally read a directory and couldn't figure out why my program didn't give any output!

As a sanity check, Python refuses to do this:

>>> f = open("/Users/jakobnissen/Documents")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IsADirectoryError: [Errno 21] Is a directory: '/Users/jakobnissen/Documents'

[aviks]

Surprisingly I hit the same issue yesterday. In my case, the open was hidden behind a file parser, which also did not error on empty input, but returned an empty object instead. So that was even harder to debug.

[StefanKarpinski]

This should definitely error.

[vtjnash]

We used to have an fstat test here, but it was slow (#35907)

[jakobnissen]

That is unfortunate. Perhaps we could check if it's a directory iff the file is size zero? I think a zero-size file can be determined when filling the initial buffer, so it should be very fast. It will, of course, slow down opening zero-size files that are actually files.

[PallHaraldsson]

You can do e.g.:

julia> open("/dev/null")   # would the check conflict with something, e.g. in /dev/input/

[KristofferC]

Consequence:

julia> using TOML

julia> TOML.parsefile(homedir())
Dict{String, Any}()

[staticfloat]

I'm pretty sure this is the branch that needs a special case: https://github.com/JuliaLang/julia/pull/35925/files#r509678679

[fredrikekre]

Just a note that some filesystems (this is CephFS) allows reading directories:

$ cat bin/
entries:                      8
 files:                       7
 subdirs:                     1
rentries:                  1538
 rfiles:                   1425
 rsubdirs:                  113
rbytes:                44240688
rctime:    1618950776.329878379

$ julia -q
julia> open("bin") do dir
           println(read(dir, String))
       end
entries:                      8
 files:                       7
 subdirs:                     1
rentries:                  1538
 rfiles:                   1425
 rsubdirs:                  113
rbytes:                44240688
rctime:    1618950776.329878379

[StefanKarpinski]

This whole situation still seems pretty unfortunate to me. @vtjnash, how big a performance hit would it be to reinstate the explicit check for whether the thing we're opening is a directory and erroring when it is?

[giordano]

Perhaps we could check if it's a directory iff the file is size zero?

But directories don't have zero size according to fstat

[cmcaine]

I think it's different notions of file size, Mose: e.g. filesize(open("/")) == 122 for me, but eof(open("/")) == true. Assuming eof or a lower level version of it are fast enough, that could be a trigger to do an extra conditional check to see if the thing is a directory.

And on file systems where directories are readable, Julia will read them, which is probably(?) the correct behaviour?

It's what cat does anyway: error when reading a directory on normal filesystems, print the directory on filesystems where directories are readable.