Skip to content
This repository has been archived by the owner on Aug 12, 2024. It is now read-only.

Writing to a file should be atomic #88

Closed
calebdoxsey opened this issue Mar 11, 2016 · 3 comments
Closed

Writing to a file should be atomic #88

calebdoxsey opened this issue Mar 11, 2016 · 3 comments
Labels

Comments

@calebdoxsey
Copy link

Currently it's possible for downstream readers to read a partially written file because WriteFile is not atomic. Ideally an updated file would only change once, and the change would be total, so that you'd either read the old state or the new state, but nothing in between.

One way to implement this is by writing to a temporary file and then renaming the file:

func safeWriteFile(dst string, data []byte) error {
    tmp := dst + ".tmp"
    defer os.RemoveAll(tmp)

    err := ioutil.WriteFile(tmp, data, 0755)
    if err != nil {
        return err
    }
    return os.Rename(tmp, dst)
}

This works because renaming a file in unix is atomic.

One thing to keep in mind that is that it's only atomic on the same physical device. So writing to /tmp may not work.

@darron
Copy link
Contributor

darron commented Mar 11, 2016

@Arkelenia and I were just talking about to this earlier today.

Am looking at getting a patch in shortly.

@darron
Copy link
Contributor

darron commented Mar 11, 2016

Right now ioutil.WriteFile truncates the file first - so that's likely what you're seeing.

Our main use has been with dnsmasq which doesn't read the file until you send a SIGHUP so we haven't been hitting this.

@darron darron added the bug label Mar 11, 2016
darron added a commit that referenced this issue Mar 11, 2016
…me the file.

This helps to make sure that processes are watching the file never see a 0-length
file in the middle of an update.

This is being done because ioutil.Writefile truncates any existing file in place
which at high velocities means bad things can happen.

This makes sure that the file is replaced atomically.
This also means we can remove the chmod code we added here:
#78
Since we're always creating a new file - the perms will always be correct.

Original bug: #88
@darron darron closed this as completed Mar 11, 2016
@darron
Copy link
Contributor

darron commented Mar 11, 2016

Fixed. Will test in staging.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants