Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use stable_logical_sequence_number? #2

Closed
bonsairobo opened this issue Aug 6, 2022 · 1 comment
Closed

How to use stable_logical_sequence_number? #2

bonsairobo opened this issue Aug 6, 2022 · 1 comment

Comments

@bonsairobo
Copy link

bonsairobo commented Aug 6, 2022

The docs say it can be used in logs to avoid double-recovery, which makes sense. But what value should actually be written into a log? I assume it can't be the stable LSN, since by definition of writing it into a log, it's not stable in marble yet. So is it just stable LSN + 1?

I can't tell if it's safe to use +1 since there might be concurrent commits and checkpoints, meaning the checkpoint into marble might bump the LSN before the commit that's using it can finish, and then suddenly it's not correct.

@spacejam
Copy link
Member

I've also found this to be error-prone, and I've removed this method from the storage rewrite that I'm about to publish. While Marble could provide out-of-band metadata storage, I'd like to keep it extremely simple if possible, and layer complexity in modular components in other systems. One way to use Marble is to write to an object with each batch that represents the last log index that you have included in an atomic writebatch to Marble. Maybe u64::MAX etc... (0 is about to become a valid object ID in the release that may happen in the next few days, too). Then when you recover your system, you just read that object from Marble that represents the last written log index, and apply anything else from your log that is required for your system's recovery based on that information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants