Implement knit as an "undo" for chop

[matthew-levan]

knit

urbit knit <pier> <input files> implements the inverse of chop¹.

knit allows pilots who have previously chopped their event log to merge their set of chopped event logs (files like data.mdb.bak.<first>_<last>) back into their pier's normal event log (data.mdb).

knit gracefully stops the given pier (if running) and verifies that all input files exist, are readable, are valid LMDB database files, and match the metadata of the specified pier. Next, it reads the keys from all the input files (and the pier's current event log) and ensures that the resulting merged set of event numbers is valid (i.e., monotonically increasing integers starting from 1)². Then, it streams the events from the input files into a fresh database file and, upon success, backs up the pier's current event log (to data.mdb.bak.<first>_<last>, as in chop) and renames the temporary database file to data.mdb.

Pilots are then free to move, archive, or delete their old .bak files, resume normal operation of their ship, and bear the burden of a higher monthly hosting fee.

Footnotes

1. See chop: offline event log truncation #165. ↩

2. Should it be necessary for pilots to provide all previous events, all the way back to 1? ↩

[belisarius222]

Ideally this would not require the user to have events all the way back to event 1. If the user has already chopped the event log previously and deleted the older events to free up disk space, it should still be possible to do a second chop and then undo it with knit if something goes wrong. The requirement of starting from the first event would prevent this.

Checking contiguity of events is needed to prevent creating an event log with an invalid state, but it should be able to start at an arbitrary event.

[matthew-levan]

Closing due to de-prioritization. chop is already trivially reversible.