fs.rename for atomic writes

[jldec]

Context

MESDK-126 (opral/inlang-message-sdk#86)

The sdk needs a simple and reliabe way to write to an existing file on disk when operating in non-browser environments like the cli and Sherlock.

Node fs.writeFile is not atomic. This exposes us to partial writes e.g. if a process exits or if there are concurrent writes to the same file, which can leave the file in a corrupted state.

We currently use a global lock to avoid concurrent writes by cooperating sdk tasks, but this is heavy and doesn't scale to larger numbers of files. It also doesn't block 3rd party writers.

As we build more sophisticated multi-file storage for v2 persistence, having a simpler atomic write capability will also allow us to explore more options e.g. for persisted secondary indexes, without resorting to heavy global locks for everything.

Proposal

extend NodeishFs with fs.rename

This would allow the sdk to write store files in 2 phases, similar to npm/write-file-atomic

1. write temp file
2. rename temp file -> store file

alternatives:

• bypass lix when not in browser and only use NodeishFs for browser apps (nope)
• detect corruption and add a recovery mechanism (complex)
• ignore the issue and use git recovery (looks bad!)
• pray for no file corruption because of partial writes 😇
• ?