Consistency of read() and write() functions.

[samoconnor]

Below is review of current write an read variants in Base.

I propose:

• deprecate readbytes and readall, replace with read and readstring respectively
• add filename-as-1st-arg support everywhere

Rationale:

• "bytes" is the default, write is not writebytes so readbytes should just be read.
• readall is named as "read_[how much]" but behaves like "read[as type]_" so it should be readstring.
• filename-as-1st-arg is generally handy, see examples in write(filename::AbstractString, data) #14546

Deprecation and addition indicated by strikethrough and bold below.
Filename = has read(filename::AbstractString...) method.

Function	Filename
write
write_[format]_
writecsv	yes
writedlm	yes
writemime

Function	Type	Filename	Whole File	Non Blocking
read_[how much]_
read(io,T)	T
readavailable	Array{UInt8}			yes
readuntil	String
readline	String
readall	String	yes	yes
read_[as type]_
readbytes	Array{UInt8}		yes
read	Array{UInt8}		yes
readstring	String	yes	yes
readlines	Array{String}		yes
readcsv	Array{T}	yes	yes
readdlm	Array{T}	yes	yes
readdir	Array{String}	yes	yes
readlink	String	yes	yes
read_[and then]_
readchomp(x)	= chomp(readall(x))	yes	yes

Related:

• Convenience functions for dealing with .gz files BioJulia/Libz.jl#12 -- Convenience functions for dealing with .gz files

• 14546 write(filename::AbstractString, data)

[nalimilan]

Thanks. I wonder whether readall should even be changed to read(String, ...).

[JeffBezanson]

Thanks @samoconnor, this is a very good design review.

I still kind of like the idea of tofile("a.dat",write)(data), since it doesn't require multiple functions to have whole-file methods.

[samoconnor]

@JeffBezanson, tofile is intriguing, but I'm not sure that I completely follow your intention. Are you suggesting that writecsv(filename, data) would be deprecated in favour of tofile(filename, writecsv)(data)? Also why not tofile(filename, write, data)? What is the advantage of the intermediate anonymous function?

[samoconnor]

Following up on what I proposed above, the new methods would be...

      read(io::IO) = readbytes(io)
readstring(io::IO) = readall(io)

     write(filename::AbstractString, d...) = open(io->write(io, d...),     filename, "w")

     read!(filename::AbstractString, a)    = open(io->read!(io, a),        filename)
      read(filename::AbstractString, t...) = open(io->read(io, t...),      filename)
 readuntil(filename::AbstractString, c...) = open(io->readuntil(io, c...), filename)
  readline(filename::AbstractString)       = open(readline,                filename)
 readlines(filename::AbstractString)       = open(readlines,               filename)
readstring(filename::AbstractString)       = open(readstring,              filename)

Unless anyone objects, I'll close #14546 and open a new PR with these methods.

[bicycle1885]

write(filename...) would be handy, but I bet people will accidentally make and/or overwrite files without noticing it.

[JeffBezanson]

Yes tofile(filename, write, data) would be just as good; for some reason my first instinct was that currying would be useful but perhaps it's not.

The "core" output functions seem to be those that write data to a stream at the current position. From there, I often want to either write a whole file in one go (like you), or get the output as a string or byte vector. I feel like there should be a standard way to add those behaviors, so that you don't need to rely on every I/O function adding a "whole file" method. open and sprint do this for files and strings, but my gripes with them are (1) calling them is too verbose, (2) the names aren't great, particularly sprint, (3) they don't generalize e.g. to getting a byte vector.

writecsv is weird because you almost always want to write a whole file. But ideally yes I would replace it with something more general. One might even want an abstraction that describes the format of the file, e.g. DelimFile(name, delim='\t', eol="\n") etc.

[samoconnor]

I think the abstractions are great as a tool for keeping the API implementation simple and efficient, but I think the API should have concrete functions that let you tell the computer to do what you want it to. writecsv is built on top of writedlm, but I'm sure that for a bunch of people who want to get their array across into Excel, they don't need to know or care about that.

As it stands if I say "Computer, put my data in a .CSV file", it says "done!",
but if I say "... put my data in /tmp/foo" it says "Well, actually it would suit me better if you opened a handle that points to /tmp/foo first, then ask me to write your data to the handle".
Sometimes it is just the API designer's job to add the little convenience methods that make the behaviour consistent across functions. .... what's that? oh, the computer is interrupting me to say that with that thing it was saying before about the file handle, it wants to remind me to close it after I'm done. Damn computer!

Another thing, it occurs to me that part of the unnecisary verboseness of open(io->write(io, data), filename, "w") is the "w" part. Maybe: the default should be "a+" and there should be create([f], filename) = open([f], filename, "w+") for the common case where the truncate is wanted. Arrays and Dicts don't have a read/write permission scheme getting in the way of ease of use, why files?

[samoconnor]

What about merging tofile and create...

create(filename::String) = open(filename, "w+")
create(f::Function, filename::String) = open(f, filename, "w+")
create(filename::String, f::Function, data...) = create(io->f(io, data...), filename)
write(filename::String, data...) = create(filename, write, data...)

create("/tmp/foo.csv", writecsv, results)
create("/tmp/foo.png", write, plot(results))

But I think I still prefer to do...

writecsv("/tmp/foo.csv", results)
write("/tmp/foo.png", plot(results))

[JeffBezanson]

All good points. I agree the "w" feels unnecessary, and to that I'd add the io-> and io, parts, since the first argument almost always has the form io->f(io, xxx).

I agree a certain set of convenience functions is unavoidable. I'm just a bit leery of expecting every I/O function f to also define

f(filename::AbstractString, x...) = open(io->f(io, x...),      filename)

[StefanKarpinski]

If we had some sort of scoped open/close construct this would be sufficiently concise that it might not even be necessary to define the f(filename::AbstractString, x...) = open(io->f(io, x...), filename) method for everything since you could just do f(open(filename), x...) and the close of the file would be automatically inserted upon exiting the scope. Of course, we'd need some kind of syntactic indication that this is happening.

[nalimilan]

Any opinions about whether readall(x) (readstring(x) in #14660) could simply be read(x, AbstractString) (and soon read(x, String) after Stefan's rework)?

This would give a much more consistent API. More generally, this would mean read(x) is just calling the method read(x, type=Vector{UInt8}) (replacement for readbytes as in #14660). We could also allow e.g. read(x, Vector{Int}) at some point if there's a need for it.

[tkelman]

Many of the specify-an-output-type API's take the type first, or maybe second after the output array for in-place methods? We've got a few possibly-overlapping conventions for first arguments.

[nalimilan]

Indeed, this is similar to the discussion at #14412. Given the conclusion there, I would say it's better to keep the output type as second argument here too.

[johnmyleswhite]

I like the idea of specifying the output type as a type rather than encoding it in the function names.

[samoconnor]

Now that #14699 and #14660 are merged, most of what this issue addresses is resolved.

A quick scan of the comment history yields the following loose ends...

• New tofile(filename, write, data) function proposed by @JeffBezanson.
• @nalimilan suggested readstring could be read(x, String) "after Stefan's rework".
• Read/write as default mode for open(filename) and create(filename) = open(filename, "w+") for cases where truncation is desired. (I've opened new issue RFC: Deprecate open(filename) mode strings ("r", "w+", etc...) ? #14844 to cover this).

I'll leave it to others to create new issues for the other items as they see fit.

[nalimilan]

@nalimilan suggested readstring could be read(x, String) "after Stefan's rework".

Actually, I think it can be changed immediately to read(x, UTF8String/ASCIIString). It's just that it will be even shorter once we have a default type named String.