Skip to content

Commit

Permalink
call fsync after flush (Refs #573)
Browse files Browse the repository at this point in the history
short: flushing the bufio buffer is not enough to ensure data
consistency.

long:
Saving an entry to the WAL calls writeLine to append data to the
autofile group backing the WAL, then calls group.Flush() to flush that
data to persistent storage. group.Flush() in turn proxies to
headBuf.flush(), flushing the active bufio.BufferedWriter. However,
BufferedWriter wraps a Writer, not another BufferedWriter, and the way
it flushes is by calling io.Writer.Write() to clear the BufferedWriter's
buffer. The io.Writer we're wrapping here is AutoFile, whose Write
method calls os.File.Write(), performing an unbuffered write to the
operating system, where, I assume, it sits in the OS buffers awaiting
sync. This means that Wal.Save does not, in fact, ensure the saved
operation is synced to disk before returning.
  • Loading branch information
melekes committed Sep 21, 2017
1 parent 2460823 commit d71d139
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion autofile/group.go
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,11 @@ func (g *Group) WriteLine(line string) error {
func (g *Group) Flush() error {
g.mtx.Lock()
defer g.mtx.Unlock()
return g.headBuf.Flush()
err := g.headBuf.Flush()
if err == nil {
err = g.Head.Sync()
}
return err
}

func (g *Group) processTicks() {
Expand Down

0 comments on commit d71d139

Please sign in to comment.