Integrate bottomless WAL: automatic replication to S3-compatible storage#92
Closed
psarna wants to merge 8 commits into
Closed
Integrate bottomless WAL: automatic replication to S3-compatible storage#92psarna wants to merge 8 commits into
psarna wants to merge 8 commits into
Conversation
psarna
force-pushed
the
bottomless_integration
branch
10 times, most recently
from
a20b555 to
27a13f9
Compare
The submodule contains source code for bottomless WAL implementation. It's shipped as a submodule, because it also makes sense to keep the implementation as a standalone repository. Namely, when integrating with Rust, it's more efficient to load the extension directly from Rust code instead of linking with a dynamic library that contains another (redundant) copy of Rust runtime.
From now on, configuring the project with ./configure --enable-bottomless-wal will compile in built-in support for S3-compatible WAL implementation which automatically replicates data to remote storage and is able to restore it.
psarna
force-pushed
the
bottomless_integration
branch
from
27a13f9 to
05c30f6
Compare
psarna
force-pushed
the
bottomless_integration
branch
from
05c30f6 to
a0108ef
Compare
psarna
force-pushed
the
bottomless_integration
branch
from
7b85027 to
e2a7477
Compare
Collaborator
|
Closed as obsolete. |
|
Hi folks! fyi this is still linked from the homepage https://libsql.org/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This patchset integrates libSQL with https://github.com/libsql/bottomless - an optional extension which adds transparent automatic replication to S3-compatible storage. It only works in WAL mode, because the implementation is based on virtual WAL.
In order to compile and link the extension statically, configure the project with:
# remember about a one-shot additional step: git submodule update --init ./configure --enable-bottomless-walThis extension is added in a form of a git submodule, because https://github.com/libsql/bottomless is its own project that also comes with a Rust library and command-line interface. Rust applications should prefer registering the extension directly in Rust code instead of compiling the support into
libsqlite3.so. The procedure will eventually get thoroughly explained in the docs.Draft, because it's in early stage of testing, with trace amount of automatic test coverage and not enough docs.