Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement soft delete #395

Merged
merged 1 commit into from
Aug 24, 2023
Merged

Implement soft delete #395

merged 1 commit into from
Aug 24, 2023

Conversation

benbjohnson
Copy link
Collaborator

This pull request adds support for soft deleting databases. Previously, deleting a database would wipe it from the primary and then the primary would send a "drop db" frame to the replicas to have them delete as well.

However, this is problematic when connected to LiteFS Cloud (LFSC) for several reasons. First, LFSC acts as the data authority so when the primary removes the database, LiteFS detects that it exists on LFSC and downloads a snapshot, thus negating the delete. Second, LFSC aims to provide durability even in the face of accidental deletion so a delete should not remove old versions of the database.

Changes to LTX (superfly/ltx#46) were made to have a zero-length database represent a deletion. These changes have been integrated into LFSC and this PR integrates it into LiteFS.

/cc @gc-victor

Fixes #382

Usage

# Create a new database on the LiteFS mount.
$ sqlite3 /litefs/db "CREATE TABLE t (x)"

# The new database has a TXID of 1.
$ cat /litefs/db-pos 
0000000000000001/f83db67898d45151

# Removing the database will delete all associated files and will be TXID 2.
$ rm /litefs/db

# The position file will not be visible via "ls" but can be read via "cat"
$ cat /litefs/db-pos 
0000000000000002/8000000000000000

# Re-creating the database will start a new database file. This is TXID 3.
$ sqlite3 /litefs/db "CREATE TABLE my_new_table (y)"

$ cat /litefs/db-pos 
0000000000000003/e01f61ed0d81644e

@benbjohnson benbjohnson added the enhancement New feature or request label Aug 24, 2023
@benbjohnson benbjohnson merged commit 837e759 into main Aug 24, 2023
16 checks passed
@benbjohnson benbjohnson deleted the ltx-zero-commit branch August 24, 2023 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve deletion via LTX zero commit
1 participant