Offer durable file writing functions

[nh2]

rio should offer file writing functions that are durable.

That involves fsync()ing the file itself after writing and before close(), and fsync()ing the directory that contains the file. It also involves failing hard on errors in fsync(), as retrying fsync has no effect.

Sources for fsyncing the directory:

• http://blog.httrack.com/blog/2013/11/15/everything-you-always-wanted-to-know-about-fsync/

See also:

• https://wiki.postgresql.org/wiki/Fsync_Errors ("fsyncgate")
• Mailing list entry PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS
• https://stackoverflow.com/questions/42434872/writing-programs-to-cope-with-i-o-errors-causing-lost-writes-on-linux from the same author
• My question about fsync() after close() and re-open on the linux-fsdevel mailing list
• My Stackoverflow answer on this topic
• The "atomic move" pattern implemented with rename() isn't enough to ensure durability
• Interesting post on the Linux side of things

Note it may not be possible to provide durable variants for all operating and file systems, but these functions should be as durable as they can be.

We should also discuss whether durability should be the default in writeFileBinary and writeFileUtf8.

In general, rio should expose durable and non-durable variants of file writing operations.

[nh2]

We should also discuss whether durability should be the default in writeFileBinary and writeFileUtf8.

My personal take right now is safe defaults are better, even if they are much slower due to the fsyncs and moves.

And ideally the docs should point out things like "if you do not need durability, use this other function which is much faster".

Also relevant for atomic moves (and performance) is whether the source and target are on the safe file system (overwise the rename() is not atomic at all).

[snoyberg]

The atomic move issue is probably the biggest question mark about this for me. AFAICT, the only way to reliably implement this is to create a temporary file in the same directory as the target file and then move it. However, in the case of program crashes/SIGKILLs/etc, there's a possibility that the file could be left behind, meaning that this change doesn't just trade performance for durability, but also introduces slight externally-visible semantic changes.

[nh2]

I found another very useful thread that discusses these issues, and particular wheter fsync() after rename() is necessary (the answer there is yes):

• https://groups.google.com/forum/#!topic/comp.unix.programmer/AM2V83RCOVE

[lehins]

Just stumbled on this ticket. It seems like it can be closed, no?

[snoyberg]

I think so