Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement knit as an "undo" for chop #184

Closed
matthew-levan opened this issue Jan 31, 2023 · 2 comments
Closed

Implement knit as an "undo" for chop #184

matthew-levan opened this issue Jan 31, 2023 · 2 comments

Comments

@matthew-levan
Copy link
Contributor

matthew-levan commented Jan 31, 2023

knit

urbit knit <pier> <input files> implements the inverse of chop1.

knit allows pilots who have previously chopped their event log to merge their set of chopped event logs (files like data.mdb.bak.<first>_<last>) back into their pier's normal event log (data.mdb).

knit gracefully stops the given pier (if running) and verifies that all input files exist, are readable, are valid LMDB database files, and match the metadata of the specified pier. Next, it reads the keys from all the input files (and the pier's current event log) and ensures that the resulting merged set of event numbers is valid (i.e., monotonically increasing integers starting from 1)2. Then, it streams the events from the input files into a fresh database file and, upon success, backs up the pier's current event log (to data.mdb.bak.<first>_<last>, as in chop) and renames the temporary database file to data.mdb.

Pilots are then free to move, archive, or delete their old .bak files, resume normal operation of their ship, and bear the burden of a higher monthly hosting fee.

Footnotes

  1. See chop: offline event log truncation #165.

  2. Should it be necessary for pilots to provide all previous events, all the way back to 1?

@matthew-levan matthew-levan changed the title Implement knit to "undo" chop Implement knit as an "undo" for chop Jan 31, 2023
@belisarius222
Copy link
Contributor

Ideally this would not require the user to have events all the way back to event 1. If the user has already chopped the event log previously and deleted the older events to free up disk space, it should still be possible to do a second chop and then undo it with knit if something goes wrong. The requirement of starting from the first event would prevent this.

Checking contiguity of events is needed to prevent creating an event log with an invalid state, but it should be able to start at an arbitrary event.

@matthew-levan
Copy link
Contributor Author

Closing due to de-prioritization. chop is already trivially reversible.

jalehman pushed a commit that referenced this issue Feb 9, 2023
## `chop`

`urbit chop <pier>` implements a simple, offline **event log
truncation**[^1] tool.

`chop` gracefully stops the given pier (if running), backs up the
current snapshot to `<pier>/.urb/bhk`, makes sure a current snapshot
exists (i.e., is fully written to disk in `chk/*.bin` with no existing
patch files), reads the metadata and the last event from the pier's
event log, initializes a fresh event log in the `<pier>/.urb/log/chop`
directory, writes the metadata and last event from the original log into
the fresh one, renames the original event log to
`<pier>/.urb/log/chop/data_<first>_<last>.mdb.bak` where `first` and
`last` are the first and last event numbers from the event log, and
exits.

Pilots are then free to move, archive, or delete their `.bak` event log
file, resume normal operation of their ship, and enjoy the many benefits
of lowered disk pressure and any reductions in associated hosting costs.

I've tested `chop` successfully on my own planet `~mastyr-bottec`
(multiple times), three different comets (all fresh), and multitudes of
fake galaxies.

Resolves #122.

Note: `knit`, which is the "undo" button for `chop`, is being
implemented in its own PR #184.

[^1]: https://roadmap.urbit.org/project/event-log-truncation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants