Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: os: add ReadDir, ReadFile, WriteFile, CreateTemp, MkdirTemp & deprecate io/ioutil #42026

Open
rsc opened this issue Oct 16, 2020 · 4 comments
Labels
Milestone

Comments

@rsc
Copy link
Contributor

@rsc rsc commented Oct 16, 2020

io/ioutil, like most things with util in the name, has turned out to be a poorly defined and hard to understand collection of things.
As part of #40025, we migrated Discard, NopCloser, and ReadAll to package io.
(They remain in io/ioutil for compatibility, of course, but new code should prefer the ones in io.)
What's left is a few OS file system helpers: ReadDir, ReadFile, TempDir, TempFile, and WriteFile.
I propose we migrate all of these to package os, with a few adjustments.
(Again, they would remain here for compatibility, but new code would prefer the ones in os.)

At that point, io/ioutil would become entirely deprecated.

The signatures would be:

package os

// ReadDir reads the named directory,
// returning all its directory entries sorted by filename.
func ReadDir(name string) ([]DirEntry, error)

// ReadFile reads the named file and returns the contents.
// A successful call returns err == nil, not err == EOF.
// Because ReadFile reads the whole file, it does not treat an EOF from Read
// as an error to be reported.
func ReadFile(name string) ([]byte, error)

// WriteFile writes data to the named file, creating it if necessary.
// If the file does not exist, WriteFile creates it with permissions perm (before umask);
// otherwise WriteFile truncates it before writing, without changing permissions.
func WriteFile(name string, data []byte, perm fs.FileMode) error

// MkdirTemp creates a new temporary directory in the directory dir
// and returns the pathname of the new directory.
// The new directory's name is generated by adding a random string to the end of pattern.
// If pattern includes a "*", the random string replaces the last "*" instead.
// If dir is the empty string, TempFile uses the default directory for temporary files, as returned by TempDir.
// Multiple programs or goroutines calling MkdirTemp simultaneously will not choose the same directory.
// It is the caller's responsibility to remove the directory when it is no longer needed.
func MkdirTemp(dir, pattern string) (string, error)

// CreateTemp creates a new temporary file in the directory dir,
// opens the file for reading and writing, and returns the resulting file.
// The filename is generated by taking pattern and adding a random string to the end.
// If pattern includes a "*", the random string replaces the last "*".
// If dir is the empty string, TempFile uses the default directory for temporary files, as returned by TempDir.
// Multiple programs or goroutines calling CreateTemp simultaneously will not choose the same file.
// The caller can use the file's Name method to find the pathname of the file.
// It is the caller's responsibility to remove the file when it is no longer needed.
func CreateTemp(dir, pattern string) (*File, error)

Notes:

  • os.ReadDir returns []DirEntry, in contrast to ioutil.ReadDir's []FileInfo.
    (Providing a helper that returns []DirEntry is one of the primary motivations for this change.)

  • os.ReadFile is identical to ioutil.ReadFile.

  • os.WriteFile is identical to ioutil.WriteFile.
    I looked into whether the permission bits should be dropped, but Go code in the wild uses a variety of popular settings (for example: 0666, 0644, 0600, 0700, 0777).

  • os.MkdirTemp is identical to ioutil.TempDir.
    It cannot be named os.TempDir because that already exists and returns the default temporary directory.
    MkdirTemp seems like it makes clear that the directory is being created and will appear next to Mkdir in docs.
    I looked into whether the directory argument should be dropped, but about 20% of calls in the wild don't use the empty string.

  • os.CreateTemp is identical to ioutil.TempFile.
    It could be named os.TempFile, but calling it os.CreateTemp is consistent with os.MkdirTemp, makes clear that the file is being created (using TempFile for this would be a function surprisingly different from TempDir), and will appear next to Create in docs.
    I looked into whether the directory argument should be dropped, but about 25% of calls in the wild don't use the empty string.

@rsc rsc added this to the Proposal milestone Oct 16, 2020
@gopherbot gopherbot added the Proposal label Oct 16, 2020
@bcmills
Copy link
Member

@bcmills bcmills commented Oct 17, 2020

Would it make sense to put ReadDir, ReadFile, and WriteFile in io/fs instead of os?
It sees like they could plausibly work with any filesystem, not just the os one.

I would argue that they fit naturally with the new Walk API, wherever that ends up: ReadDir, ReadFile, WriteFile, and Walk all share the property that they implement a high-level operation in terms of existing lower-level operations.

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 17, 2020

(MkdirTemp and CreateTemp seem like they belong in os, though: the location of the temporary directory is OS-specific, and CreateTemp, at least, should return a *os.File rather than an abstract fs.File.)

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 17, 2020

Actually, even MkdirTemp and CreateTemp could belong in io/fs, if we define an FS extension interface for a method that returns a default directory for temporary files.

@rsc
Copy link
Contributor Author

@rsc rsc commented Oct 20, 2020

@bcmills, io/fs already has ReadDir and ReadFile. It would be fine to add CreateTemp and MkdirTemp if/when io/fs adds any writing operations.

But it probably wouldn't make sense to put them only in io/fs. Reading from the operating system's file systems is insanely common. For example, even if we were starting over we would certainly keep os.Open(name) rather than tell people to use fs.Open(os.Dir("/"), name) or os.Dir("/").Open(name) (neither of which would be correct on Windows anyway).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.