ForestDB: Implement database encryption

[snej]

We don't support database encryption for ForestDB-based databases because there's no ForestDB equivalent of SQLCipher. We may need to implement one ourselves, i.e. a ForestDB file I/O plugin that encrypts the file.

It looks like libSodium's Salsa20 cipher would be a good choice for this because it supports jumping to an arbitrary place in the ciphertext and deciphering from there. [see implementation notes below]

[snej]

Implementation notes:

Encryption is hooked into ForestDB's filesystem abstraction layer as a new set of filemgr_ops. The open operation compares the path with a key-registration map; if a key is found the file is treated as encrypted, and read and write operations will do the appropriate crypto.

The file is considered as a series of 4096-byte blocks. It is assumed that a read or write call will operate on exactly one entire block — this is always true when ForestDB's file cache is enabled.

A block is encrypted using AES-256 (same as SQLCipher); the key is the exact 256-bit key registered for that file. The padding is CBC, which requires that every message (block) use a different initialization vector; the IV is derived from the block number using the Encrypted Salt-Sector Initialization Value (ESSIV) algorithm. (This is the same crypto system used in the Linux kernel as part of the dm-crypt disk encryption feature.)

I had to implement a kludge to support database compaction. The automatic compactor creates a temporary file that it writes the live documents to, then replaces the original file with the new one. Unfortunately it isn't possible at the file-I/O layer to tell that the temporary file is associated with the original one; but the new file needs to be encrypted using the same key. The kludge is to detect the naming pattern that ForestDB uses for the temporary file: it appends a suffix consisting of a period and one or more digits. When the open call sees this pattern, it looks for an already-encrypted open file with the same base name and uses that file's encryption key.

I'm discussing a better solution with the ForestDB team; it will require some changes to ForestDB itself, which will hopefully be minor. For now, the kludge is reliable enough.